Factors Behind Regional Inequality in in CAP Paper 224, July 2019

Aigerim Kopeyeva was a Visiting Fellow at the George committed to the education of all Washington ’s Program in Spring citizens. As Lucas put it, if one considers the consequences for 2019. She holds an MA in Education Management from human welfare, it is hard to think King’s College London. In recent years, she has been about anything else.5 Although working on educational issues in Kazakhstan. Lucas was talking about nation- al-level performance, wondering “I do not see how one can look ing quality of education at nation- if one country’s positive econom- at figures like these without al level. Yet when disaggregated ic experience might be applied to seeing them as representing by region, language, or urban/ another, I find his words stagger- possibilities. […] The rural residence, the data from ingly appropriate for the situa- consequences for human welfare TIMSS and other large-scale as- tion in Kazakhstan. involved in questions like these sessments paint a bleaker picture. are simply staggering: once one Once we start to analyze a coun- starts to think about them, it is There is a knowledge and skill try’s economic performance, we hard to think about anything gap of several years between stu- inevitably circle back to the qual- else.” dents from different regions of ity of education. Since school is— —Lucas, 1988 Kazakhstan. According to the re- and will long remain—the single sults of the OECD’s Programme institution where a citizen spends n the fall of 2016, Kazakhstan for International Students As- the longest period of his/her life, was building momentum in sessment (PISA), most recently it has both the highest privilege of the international educational administered in Kazakhstan in and the biggest burden of respon- I 2015, 15-year-olds in the West arena. For the first time, the sibility for his/her “formation.” country’s school students ranked and South of the country (, in the top 10 internationally for Mangystau, South Kazakhstan, In this paper, I analyze regional their performance in math and and oblasts) are at least student achievement in Kazakh- science, according to the Trends two years behind their peers in stan from the basis that it is vital in Mathematics and Science Almaty city in reading, math, for regional and national human Study (TIMSS).1 Kazakh fourth- and science. As a national report capital development and prede- and eighth-graders demonstrated on the PISA results shows, half termines a country’s potential ability and knowledge compara- of 15-year-olds in Atyrau oblast for economic growth. In other ble to their peers from countries are “functionally illiterate” in words, I argue that education is like Finland, South Korea, and science,3 while more than half of the single most important factor Singapore and outpaced students students in South Kazakhstan4 in developing a nation’s human from Great Britain, the USA and and Mangystau oblasts are func- capital and thus determining its Australia, a reality that was disap- tionally illiterate, meaning that economic growth. Achieving pos- pointing to the latter group of they could not complete tasks of itive change therefore requires countries.2 the first level of difficulty. understanding the factors behind gaps in student achievement. This was a great achievement for This seems like an unacceptable the country and indicated the ris- reality for a unitary state that is

CAP Fellows Paper 224 1 Theoretical Framework in policy and economic discourse • “Expected Learning-Adjusted across the globe, with studies Years of School” is one of the Education, Human Capital, and showing a positive correlation three components of the Human Economic Growth between the two. Pelinescu found Capital Index (along with a positive relationship between “Survival” and “Health”). The Human capital is generally un- a country’s capacity to innovate component reflects the quantity 10 derstood as the collection of skills and its GDP per capita. Barro and quality of education.15 The and experiences that an individu- compared the potential input of measures “adjusted years of al, firm, or country possesses. The human and physical capital to schooling” and “harmonized test World Economic Forum defines economic outcomes, noting that a scores” have been developed by human capital as “knowledge and higher ratio of human to physical World Bank experts to better skills people possess that enable capital “tends to generate higher reflect the quality of schooling.16 them to create value in the global economic growth” due to both a • In its Global Human Capital 6 economic system.” country’s capacity to absorb tech- report, the World Economic nology and the fact that human Forum placed knowledge and In his seminal work “On the Me- capital is more difficult to adjust skills at the core of all four key 11 chanics of Economic Develop- than physical capital. Therefore, elements of human capital.17 ment,” Lucas formulates human “a country that starts with a high “Capacity” stands for the formal capital as the “skill level” of an ratio of human to physical capi- education level of the population, individual, linking it directly to tal—such as in the aftermath of a while “Development” measures 7 work productivity. The World war that destroys primarily phys- the formal education and Bank defines human capital as ical capital—tends to grow rapid- upskilling of the workforce and the the “knowledge, skills, and health ly by adjusting upward the quan- next generation. “Deployment” 12 that people accumulate over tity of physical capital.” and “Know-How” measure their lives, enabling them to re- adults’ application of their skills alize their potential as productive The OECD views education as and the depth of specialized skills 8 members of society.” central to human capital devel- that labor market participants opment, observing a correlation possess. Today, there are several inter- between education level, GDP • The Global Competitiveness national surveys performed by per capita, and overall economic Index (GCI), an annual survey 13 major analytical institutions growth. Education is often used of cross-country competitiveness measuring the quality of human as a proxy for human capital in also performed by the WEF, capital worldwide. These include studies of the effect of human cap- analyzes countries’ performance the UNDP’s Human Develop- ital on economic growth. Blundell on 12 main pillars, including ment Index (HDI), the World et al., for example, identify three “skills.”18 Economic Forum’s Global Com- main components of “human petitiveness Index (GCI), and capital”: early ability (acquired or On all of these rankings, educa- the World Bank’s Human Cap- innate); qualifications and knowl- tion is represented by two main ital Index (HCI). In addition to edge acquired through formal ed- indicators: the highest level of rankings, such studies provide ucation; and skills, competencies, education obtained by the popu- thorough analysis of human capi- and expertise acquired through lation (or years of schooling) and 14 tal components and the trends in on-the-job training. educational outcomes in math, their development. science, and reading as measured Thus, there seems to be a strong by international large-scale as- As World Bank experts note, “a sense that there is a direct cor- sessments (ILSAs). country’s human capital is crit- relation between education and ical for its economic success,” the country’s overall economic As OECD experts note, all eco- comprising 64 percent of a na- performance. In fact, education nomic growth theories “see edu- 9 tion’s wealth. The notions of parameters are present in every cation as having a positive effect human capital and economic major human capital index: on growth.”19 Barro and Lee’s growth are tightly intertwined database of international data on CAP Fellows Paper 224 2 average years of schooling is of- to be the single most influential boys. Blundell et al. have found ten used as a point of reference variable explaining variations in that the average annual return for when calculating the effect of ed- income level both between and men with a first degree is almost ucation on economic outcomes within countries.24 The authors’ two times less that for women.27 like GDP per capita.20 Return on analysis of data for over 1,500 • Finally, less economically investment in education is nor- subnational regions showed that developed countries tend to enjoy mally calculated as the ratio of education is the only factor that higher rates of economic growth costs incurred by a citizen or a explains a substantial share of re- than developed ones (due to the state to his/her earnings at the gional variation.25 diversity of needs and spending given moment.21 in more developed countries).28 Thus, I do not plan to go into The effect of education on coun- much detail on this aspect. It is, Researchers often refer to large- tries’ economic performance has however, worth mentioning sev- scale studies like PISA (Pro- been confirmed by multiple stud- eral major trends that can be ob- gramme for International Student ies, performed both by larger in- served from these studies: Assessment), TIMSS (Trends in stitutions like the World Bank International Mathematics and and by individual researchers.22 • There is strong evidence Science Study), or PIRLS (Prog- In the United Kingdom, for ex- that primary education brings a ress in International Reading ample, it has been found that cit- higher return on investment than Literacy Study) to calculate re- izens with formal qualifications secondary or higher education. turn on investment in education. “have significantly larger returns This is due to the foundational These studies provide the most than individuals with the same quality of the first years of thorough and representative da- number of years of schooling but education, when an individual’s tabase on international academic who completed no formal qualifi- cognitive abilities are formed. performance. cation.”23 As the effect of education is cumulative, the returns on There are plausible concerns about In their analysis of the determi- secondary or higher education whether it is entirely appropriate nants of economic development (any additional year of training) to use the tests to measure in 110 countries, Gennanioli et tend to be smaller.26 quality of schooling—that is, the al. have tested the effect of ge- • Return on investments in girls’ extent to which absolute learning ography, education, institutions, education are higher than those outcomes represent school and culture, finding education on investments in educating processes and the factors that

Box 1. International Large-Scale Assessments o PISA is administered by the OECD and tests 15-year-olds’ competencies in reading, science, and math. The assessment takes place once every 5 years, with roughly 45-50 participant countries in each round. The OECD does not rank the countries on the basis of their results, instead preferring to group them into broader clusters. However, the test scores allow countries to determine their posi- tions in relation to other countries. o TIMSS and PIRLS are performed by the IEA once every four years. While TIMSS tests fourth- and eighth-graders’ knowledge of math and science, PIRLS targets only fourth-graders, assessing their reading literacy. The IEA’s results are more straightforward than those from the OECD in the sense of producing a clear ranking of countries on the basis of their test results. o The principal difference between the PISA and TIMSS assessments (which overlap in their coverage of subjects and also age groups) has been articulated as a difference in the areas measured. While PISA problems aim to measure functional literacy (ability to apply knowledge learned to real-life sit- uations), TIMSS provides data on the extent to which a student has effectively consumed the school curriculum.82 Both studies disaggregate their test results by level of difficulty and collect background information about factors that influence academic performance: student’s family, school infrastruc- ture, teacher training, etc.

CAP Fellows Paper 224 3 impact overall academic success. international databases, make it full-time.38 There is also a question about the possible to study the effect of a appropriateness of using “years of particular variable on educational As yet, there is not much evidence schooling” (normally the period of outcomes. or research on the possible pre-school and school education) influence of an individual’s values to measure human capital. One Harding et al. and Crede et al., on his/her academic performance of the main reservations is that for instance, studied impact of and subsequent “success in life.” such an approach automatically parental education on student Nevertheless, in this paper I place assumes that every country in achievement.32 Parents’ career a certain amount of emphasis the ranking (or every region in level was also found to influence on this aspect, as there are a country) delivers the same student achievement,33 as was indications that this might be amount and quality of learning a child’s access to educational the next most important impact to every student in one year of resources at home.34 Blundell factor behind a student’s SES. schooling, which is undoubtedly cites local environment and the not the case. According to the quality of schools in a student’s For instance, according to authors of the Human Capital neighborhood as affecting Wilson, one’s decision to proceed Index, there is “a gap in human educational attainment.35 to another level of education capital formation: students are might be influenced by perceived in school but are not learning.”29 Interestingly, a number of economic returns on schooling The OECD argues that measuring recent studies have focused and by the advantages (utilities) cognitive skills allows for a more on—and demonstrated—the an individual actually experiences objective analysis of economic direct and indirect influence of given his/her socioeconomic growth than does a mere a mother’s human capital on background. She has found that measurement of the quantity of the development of her child. both perceived economic returns school years attained.30 Harding et al. constructed a and students’ SES influence framework in which a mother their educational attainment. However, given that these possesses a certain level of social, The accomplishments of older assessments remain the primary cultural, and human capital that individuals in their neighborhood, sources of insight into academic she can transmit to her child.36 who have a similar background, achievement in the international These modes of transmission influence students’ perceptions context, they will probably may include choosing a better of the value of education, perhaps continue to be utilized for human school, finding better educational prompting them to want to capital studies for a long time yet. opportunities, her implicit acquire higher education. In Barro found that international test behavior, and even the “quality” other words, “youths respond scores have strong explanatory of her immediate social circle rationally to economic incentives power for economic growth and (friends and colleagues). in education.”39 are more accurate in predicting it than years of schooling.31 However, a mother’s commitment This utility-maximizing to work does not show such a perspective is predominantly Factors Influencing Educational clear positive influence on a shaped by the surrounding Outcomes child’s academic success or future neighborhood and/or by earnings. Wilson has found that students’ immediate circle of Students’ socioeconomic status having a mother who works full- communication. The more (SES), mostly understood as their time correlates negatively with a successful are those who have family background and exposure child’s income level in his or her proceeded to higher education to learning resources, is the early twenties.37 Muller likewise and are now working, the higher contextual factor most often seen found that children whose are the chances that a school to impact student achievement in mothers work part-time achieve graduate will follow their example. academic research. Studies like better results than their peers By contrast, if the costs of getting PISA or TIMSS, along with other with mothers who are employed higher education (including

CAP Fellows Paper 224 4 the potential loss of income Gennanioli et al. have tested Methodology compared to being employed the influence of intra-regional during this period) are too high in characteristics like geography To analyze gaps in student the eyes of a teenager or his/her (oil reserves, weather conditions, achievement between regions of parents, the young person might etc.), culture (mutual trust), Kazakhstan, I use TIMSS 2015 opt out of going to university. institutions, and education data. This calculus might also influence (educational attainment) on a student’s interest in learning human capital in more than The choice of TIMSS over while still at school—if there is 1,500 subnational regions in 110 PISA is explained, first and no point of going to university, countries.43 Interestingly, the foremost, by the structure of then doing well in school also authors found that oil reserves do the sample: TIMSS includes loses its value: as Edgerton et al. not explain any significant income only school students, whereas put it, “perceived penalties for variation within or between PISA (being tied to age) allows underperforming academically countries. Quality of institutions, college students to be included 47 may pose less of a deterrent to while accounting for 25 percent of as well. Moreover, using TIMSS lower SES students who are not inter-country income variation, data makes it possible to follow anticipating substantial return likewise does not explain intra- the methodology of the World from continued formal education, country differences in per capita Bank’s Human Capital Index, while the cost of underperforming income. which likewise uses TIMSS data for higher SES students may be to calculate the achievement gap 48 perceived more intensely.”40 Meanwhile, regional education between countries. Finally, explained 58 percent of inter- TIMSS is preferable to PISA Regional development is another country and almost 40 percent due to the fact that the latter’s important factor influencing of intra-country variation in per 2015 data for Kazakhstan is 49 student achievement. Since capita income. In other words, incomplete. the economy of the region is of the factors tested, “none come reflected in the quality of life close to education in explaining My particular focus is on South of students’ families and in within-country variation in and West Kazakhstan. From their neighborhoods, regional income per capita.”44 The authors the three major international development is often seen as concluded that regional education large-scale comparative studies having an indirect impact on was “a critical determinant of in which Kazakhstan has student achievement. Edgerton regional development, and the participated recently (PISA 2015, et al., for example, assert that only such determinant that TIMSS 2015, and PIRLS 2016), it “observed interprovincial explains a substantial share of is evident that those regions have differences in academic regional variation.”45 demonstrated the lowest student proficiency are in general achievement. Moreover, it is consistent with long standing Thus, there is strong evidence of possible to see that some oblasts disparities between provinces the importance of socio-economic are underperforming across all in fiscal capacity.”41 This is factors for student achievement areas of assessment—both by echoed by Tesema and Braeken, and for the contention that discipline and by cohort. who found in their research on these factors may vary between the regional factors impacting regions of a country. The Table 3 shows the number of education in Ethiopia that primary focus of the present times that different oblasts have “whenever different regions paper is student achievement been ranked among the three within a country have major in Kazakhstan disaggregated to with the lowest performance economic differences, it is likely regional (oblast46) level. The data across all 8 assessment areas: that students from economically on Kazakhstan’s performance in math and science for fourth- and less developed regions are more recent large-scale assessments is eighth-graders in TIMSS; math, disadvantaged.”42 taken from official international science, and reading for 15-year- and national reports. olds in PISA; and reading for fourth-graders in PIRLS. CAP Fellows Paper 224 5 Box 2. Regional Performance in Kazakhstan: Context

Variation in economic performance across Kazakhstani regions is not surprising given the country’s large territory and the uneven distribution of its population. So far, regional performance in Kazakhstan appears to have been fairly understudied.

Below are some of the most recent avail- able data on regional performance and education provided by international and national rankings, along with relevant statistics.

EBRD Regional Capability Index (2015)

One attempt to study the tendencies in regional development in Kazakhstan was made by Whiteshield Partners in 2015 (funded by the EBRD and the ). They used their Regional Capability Index to group the 16 administrative units of Kazakhstan into “winning,” “stagnating,” and “losing” clusters. The key criteria were economic complexity, diversity, and industrialization in the years leading up to 2014.

Table 1. Regional performance, 201483 Winning Stagnating Losing Almaty oblast Almaty city East Kazakhstan oblast Zhambyl oblast oblast South Kazakhstan oblast North Kazakhstan oblast Akmola oblast West Kazakhstan oblast Astana city Kostanay oblast Karagandy oblast Pavlodar oblast Source: Whiteshield Partners, “Diversification of Kazakhstan’s Economy: A Capability-Based Ap- proach,” EBRD, Astana, 2015, https://www.ebrd.com/news/2015/new-study-urges-diversifica- tion-of--economy.html.

Among the main factors influencing regional capability development, the authors cited business barri- ers and quality of governance, education, and institutions. Despite the fact that Almaty oblast and South Kazakhstan oblast are the two most populated areas and have “comparable economic structure[s],” the authors acknowledge that Almaty oblast is more diversified and thus more open to innovation, while South Kazakhstan specializes in simpler manufacturing.84 IAC Ranking of Regional Education Systems (2015-2017)

The Information-Analytic Centre under the Ministry of Education (IAC) publishes an annual National Report on Education ranking the regional educational systems. The ranking provides an aggregated index of regional performance across three levels of education (primary, secondary, and higher). The index groups regional education systems into those with “very high,” “high,” “average,” and “low”

CAP Fellows Paper 224 6 effectiveness. In 2015-2017, none of the oblasts was classified as “very high” in effectiveness. Pavlodar oblast received the accolade of “highly effective” in 2015 and 2016, the only education system to do so, but in 2017 none of the regions made it into that category.85 Table 2. Effectiveness of regional education systems index (IAC)

2015 2016 2017 index rank index rank index rank Pavlodar 0.741 1 0.725 1 0.643 2 oblast East KZ 0.68 2 0.602 5 0.572 8 oblast Karagandy 0.654 3 0.608 4 0.579 5 oblast West Kazakh- 0.642 4 0.651 2 0.576 6 stan oblast Astana city 0.634 5 0.592 6 0.573 7 North Ka- zakhstan 0.61 6 0.575 9 0.621 3 oblast Aktobe oblast 0.605 7 0.528 12 0.491 12 Mangystau 0.601 8 0.509 13 0.478 13 oblast Akmola 0.6 9 0.583 8 0.544 9 oblast Almaty city 0.583 10 0.639 3 0.659 1 Zhambyl 0.57 11 0.554 10 0.581 4 oblast Kostanay 0.57 12 0.535 11 0.53 11 oblast Atyrau oblast 0.528 13 0.475 15 0.441 14 Kyzylorda 0.517 14 0.587 7 0.531 10 oblast South Ka- zakhstan 0.502 15 0.408 16 0.361 16 oblast Almaty oblast 0.485 16 0.639 14 0.432 15

Sources: Information-Analytic Centre, “Natsional’nyi doklad o sostoianii i razvitii sistemy obra- zovaniia Respubliki Kazakhstan po itogam 2015 goda,” Astana, 2016, http://iac.kz/sites/default/ files/nd-2016_po_itogam_2015_goda_kratkaya_versiya_rus.yaz._iac_zashchita.pdf; Informa- tion-Analytic Centre, “Natsional’nyi doklad o sostoianii i razvitii sistemy obrazovaniia Respubliki Kazakhstan (za gody Nezavistimosti Kazakhstana),” Astana, 2017, http://iac.kz/sites/default/ files/nacdok-2017_ot_kgk_final_09.08.2017_10.00-ilovepdf-compressed.pdf; Information-Ana- lytic Centre, “Natsional’nyi doklad o sostoianii i razvitii sistemy obrazovaniia Respubliki Kazakh- stan po itogam 2017 goda,” Astana, 2018, http://iac.kz/sites/default/files/nacionalnyy_doklad_ za_2017_god_s_oblozhkami_dlya_sayta.pdf.

CAP Fellows Paper 224 7 Whereas the regions with education systems that fall into the “average effectiveness” cluster are typically from North, East, and Central Kazakhstan, those with “low” effectiveness are usually from the Southern and Western regions of the country. Kostanay (North) and Akmola (Central) oblasts also fell into the latter group in 2017. South Kazakhstan and Almaty oblasts are continuously ranked as the least effective in terms of education; Atyrau oblast is also usually found in the bottom part of the ranking.

National Statistics • In 2018, the population of Kazakhstan reached 18 million people. The territorial distribution of the population has always been uneven. The most striking difference is between sparsely populated North and West Kazakhstan and the densely populated South. While Atyrau, Mangystau, Aktobe, and West Kazakhstan oblasts together account for 16 percent of country population, the southern part of Kazakhstan (Almaty, Zhambyl, Kyzylorda, South Kazakhstan oblasts) is home to 37 percent of the population.86 South Kazakhstan oblast alone has a population of 2.9 million people, 16 percent of the country’s total, and continues to grow. • Accordingly, as can be seen from Figure 1, South Kazakhstan accounts for the highest share of the country’s school-age population. Of the country’s over 3 million school students, one-fifth live in Turkestan oblast and Shymkent city (former South Kazakhstan oblast) and a further 13 percent live in Almaty oblast. • There are clear differences in terms of regional migration. South Kazakhstan region experiences the largest outflows—in Zhambyl oblast in 2016, the negative balance of migration doubled in comparison to 2015 to reach over 16,000 people, and it stood at over 17,000 in 2017. According to the Statistics Committee, the negative outflow in South Kazakhstan oblast reached almost 30,000 people in 2016, while in 2015 and 2017 it totaled around 14,000 people. In contrast, both Astana and Almaty cities showed a positive balance of over 30,000 people in 2017. In the Western region, however, outbound migration is minimal, with Atyrau oblast even showing a positive balance of over 100 people in 2017. Outside of the Southern region, the highest negative balance (close to 14,000 people in 2017) is observed in East Kazakhstan oblast. • Forty-three percent of Kazakhstanis live in rural areas. The largest share of the rural population is registered in Almaty oblast (77 percent in 2018). The variation in the size of the rural population in recent years is not significant (a decline of two percent in North, West, and East Kazakhstan oblasts compared to 2015, and a four percent increase in Mangystau oblast since 2015). • In 2017, the average monthly wage in Kazakhstan was KZT 150,827 (approx. US$454).87 Oil-producing Mangystau and Atyrau oblasts show the highest average monthly wage, over 70 percent higher than the national average. Meanwhile, the lowest monthly wage is observed in South Kazakhstan, North Kazakhstan, and Zhambyl oblasts, where it is around 30 percent lower than the national average. Figure 1. Distribution of school student population across Kazakhstan, 2018

Source: IAC, “Education System Statistics in the Republic of Kazakhstan,” national report, 2019.

CAP Fellows Paper 224 8 Figure 2. Rural population as a share of the regional total (percent), 2018

Source: Statistics Committee, Ministry of National Economy (2019).

Figure 3. Average monthly wage in Kazakhstan (thousand KZT), 2015-2017

Source: Statistics Committee, Ministry of National Economy (2019).

All data on regional student underperform. A striking example The rankings may not fully achievement are taken from is Mangystau oblast, which was represent the quality of national reports issued by the among the bottom three regions secondary education; they may Information-Analytic Centre, the in seven of eight assessment also be affected by out-of-school national coordinating institution areas. Students in another oil- factors. Yet both objections for international large-scale producing region, Atyrau oblast, are “features” of my argument studies. For full student showed the lowest achievement rather than “bugs” in it: we need achievement ranking tables for in five assessment areas in all to understand exactly what lies all three studies, see Appendix 1. three studies (see table 4). behind regional variations in student achievement before we It is hard to ignore the fact that It would be reasonable to argue can treat these test results as regardless of the area of study that it is inappropriate to draw indicators of education quality in (science, math, or reading) or conclusions about regional any particular region. the age cohort (fourth-graders, educational performance based eighth-graders, or 15-year- solely on such a simplistic To be dramatic about it, the role, olds), certain regions tend to interpretation of rankings. value, and status of school as an CAP Fellows Paper 224 9 Table 3. Number of times oblasts were ranked in the bottom three on the eight assessment areas of recent ILSAs Appearance Appearance Region Oblast in bottom 6 in bottom 3 (times) (times) West Kazakhstan West Kazakhstan oblast 1 0 South Kazakh- South Kazakhstan oblast 7 4 stan Almaty oblast 6 4 Atyrau oblast 8 6 West Kazakhstan Mangystau oblast 8 7 North Kazakhstan Kostanay oblast 6 2 West Kazakhstan Aktobe oblast 3 0 Central Kazakhstan Akmola oblast 6 1

Central Kazakhstan Karagandy oblast 1 0

Central Kazakhstan Astana city 2 0 Source: Author’s compilation on the basis of PISA 2015, TIMSS 2015, PIRLS 2016 national reports, Information-Analytic Centre.

Table 4. Bottom 6 oblasts across eight assessment areas of recent ILSAs TIMSS 2015 PISA 2015 PIRLS 2016 4th-grade 4th-grade 8th-grade 8th-grade Science Reading Math Reading science math science math 11 West Kaz Atyrau South Kaz Kostanay Akmola Aktobe Karagandy Astana city oblast oblast oblast oblast oblast oblast oblast 12 Akmola South Kaz Kostanay Akmola Akmola Almaty Mangystau Astana city oblast oblast oblast oblast oblast oblast oblast 13 Atyrau Akmola Aktobe Almaty Aktobe Aktobe Kostanay South Kaz oblast oblast oblast oblast oblast oblast oblast oblast 14 South Kaz Kostanay Almaty Kostanay Mangystau South Kaz Akmola Almaty oblast oblast oblast oblast oblast oblast oblast oblast 15 Almaty Almaty Mangystau Atyrau South Kaz Mangystau South Kaz Mangystau oblast oblast oblast oblast oblast oblast oblast oblast 16 Mangystau Mangystau Atyrau Mangystau Atyrau Atyrau Atyrau Atyrau oblast oblast oblast oblast oblast oblast oblast oblast

Sources: Information-Analytic Centre, “Main Results of International Study PISA 2015”, Astana, 2017, http://iac.kz/sites/default/files/nac_otchet_pisa-2015_final.pdf; Information-Analytic Centre, “Kazakh- stan’s Results in TIMSS 2015”, Astana, 2017, http://iac.kz/sites/default/files/1_nac_otchet_tims_2015_ russ_pdf_0.pdf; Information-Analytic Centre, “Kazakhstan’s Results in PIRLS 2016”, Astana, 2018, http:// iac.kz/ru/events/rezultaty-kazahstana-v-pirls-2016.

CAP Fellows Paper 224 10 institution is at stake here, score was 37 out of 80 points. the measure note, the advantage especially in regions with low of LAYS is that it reflects both achievement. Can we be sure We can see that students the quantity and the quality of that test results fairly reflect the achieved the lowest scores in schooling.51 amount of work local teachers Mangystau and Atyrau oblasts. put in to get a failing student past In South Kazakhstan and Almaty This approach was originally the threshold? Or perhaps the oblasts, ninth-graders showed used to measure differences external factors in a region are so achievement close to the national between countries. In this work, I significant that a school’s efforts average. apply it to calculate the difference merely scratch the surface—or in learning-adjusted years of worse, are made in vain? Since the age cohorts tested in schooling between 16 oblasts TIMSS (eighth-graders) and in Kazakhstan. As the authors Moreover, the answers to these EALA (ninth-graders), as well as note, “schooling is not the same questions have clear implications in PISA (15-year-olds) and PIRLS as learning.”52 While educational for educational policy. Any plans (fourth-graders) are different attainment rates (highest level for strategic improvements or (although there is a possible of education) are still widely systemic interventions should be overlap between TIMSS, PISA, used to measure the quality of grounded in certainty that a good and EALA), I do not perform educational systems, it is also initiative will not have uneven correlation analysis between clear that countries (or regions) regional effects due to varying these results. with the same expected years conditions in the regions. of schooling are hardly equal in Calculating Learning-Adjusted academic achievement. Thus, Figure 4 shows the results of Years of Schooling (LAYS) for 16 combining quality and quantity the External Assessment of Regions of Kazakhstan50 of schooling is an important step Learning Achievement (EALA) toward making assessments of administered to ninth-graders in The Learning-Adjusted Years of educational systems truer to life. 2015. This national standardized Schooling measure was recently test measured students’ developed by the World Bank to The overarching formula for knowledge in math and science assess the quality of education LAYS is: (physics, biology, geography, and under the Human Capital Index chemistry). The national average framework. As the authors of LAYSc=Sc * R n/c

Figure 4. External Assessment of Learning Achievement (EALA), 201588

Source: Information-Analytic Centre, “Analiz rezul’tatov vneshnei otsenki uchebnykh dostizhenii uchashchikhsia 9 klassov (VOUD-2015),” Astana, 2015, http://iac.kz/ru/ana- lytics/analiz-rezultatov-vneshney-ocenki-uchebnyh-dostizheniy-uchashchihsya-9-klass- ov-voud-2015.

CAP Fellows Paper 224 11 Box 3. TIMSS 2015 in Kazakhstan TIMSS is more focused on assessing students’ academic TIMSS 2015 was administered across 57 countries, with over 580,000 knowledge of the curriculum students taking part worldwide. The Kazakh sample consisted of 9,579 than on determining their ability students, 44 percent of whom were from rural areas. to apply it in real-life situations (as the OECD’s PISA does). National analysis revealed that across all , Thus, given the strong traditions students in schools where Russian is the language of instruction of science and math teaching perform better than their peers who study in Kazakh. Rural students that are still present in Kazakh significantly lag behind their urban peers, while boys and girls show secondary schools, TIMSS data nearly similar achievement levels. The highest gap in average TIMSS seem to reflect the Kazakh scores is observed between the eighth-graders instructed in Russian curriculum even more than does 54 and Kazakh, respectively, in Atyrau oblast, with the latter lagging over PISA. 120 points behind the former in math and science. Expert Survey Children whose parents have higher education score significantly higher than those whose parents have only secondary or vocational To gain a better understanding education. Similarly, if a child attended education facilities of the scope of factors affecting for 3 years of more, his/her scores are higher than those of students student achievement in regions, who attended for less than 3 years.89 I conducted a survey of experts. The aim of the survey was where Sc is equal to average Broader concerns relate to twofold: a) to understand the years of schooling in country the extent to which student level of general awareness of c and R n/c is the measure of achievement as reported by regional disparities in student average learning for the relevant large-scale studies can represent achievement; and b) to get a pool cohort and country relative to a both in-school processes and a of professional opinions on what benchmark country.53 student’s family context. In other factors underpin low academic words, do these tests measure achievement in Kazakhstan’s For the purposes of my work, actual quality of learning/ regions. I use region-level data instead schooling in a particular school? of country-level data and take While there is no ultimate The expert sample consisted Almaty city as a benchmark answer to this question, as school of 16 respondents specializing region, since it has the highest effectiveness is a constantly in education (10) and other scores in all areas but one. I developing movement in professional spheres (6). provide a detailed explanation of education science, I refer to the World Bank’s formula and my the multiple previous studies Data and Results adjustments in the Appendix. that have employed PISA and TIMSS data as the most complete LAYS Calculation Results Limitations information dataset on countries’ educational performance to date. Table 5 displays the results of It is important to mention two LAYS calculations. These are pre- possibly significant limitations In the Kazakhstani context sented by TIMSS discipline (sci- relating to the calculation of LAYS in particular, there are some ence, math) and cohort tested in this work. First of all, since concerns about the extent to (fourth- and eighth-graders). The LAYS was originally designed to which test tasks reflect the local first column shows each region’s calculate country-level results, school curriculum. However, test score, while the LAYS column there is a possibility of significant these are not particularly shows years of schooling adjusted standard error when comparing troublesome. According to the for the “quality” of learning. region-level data, which could national coordinating institution affect the final calculations. (Information-Analytic Center), Almaty city receives a score of 11

CAP Fellows Paper 224 12 in every LAYS column, as it acts school. Calculations on the as- and East Kazakhstan oblast. The as a benchmarking (numeraire) sumption that learning starts at gap in learning-adjusted years of region. However, it is also three school (first grade) can be found schooling between these regions years behind the expected 14 in Appendix 1. and Almaty city is around half a years of schooling for the coun- year. try.55 The calculations presented As can be observed from the ta- in Table 6 are made with an as- ble, the regions closest to Almaty Thus, not the entire South sumption that learning starts in city on the LAYS measure are Kazakhstan region lags sig- , leading to three Kyzylorda oblast (which even out- nificantly behind in academ- years of learning in preprima- performs Almaty city on eighth- ic achievement. However, Al- ry education and 4/8 years in grade math), Zhambyl oblast, maty oblast is more than 2 years

Table 5. Sample of experts E1* E2 E3 E4 E5 E6 E7 E8 E9 E10 E11 E12 E13 E14 E15 E16 Gender, M, F, F, F, F, M, M, M, M, M, M, F, M, M, F, M, age 56 55 26 42 70 32 50 34 35 36 29 27 34 58 36 32 *E1, E2, etc.—Expert 1, expert 2, etc. Source: Compiled by the author on the basis of expert sample. Table 6. Learning-adjusted years of schooling (LAYS) for regions of Kazakhstan

TIMSS TIMSS TIMSS TIMSS

score score score score 4th grade, LAYS 4th grade, LAYS 8th grade, LAYS 8th grade, LAYS science math science math Almaty city 623 11 608 11 591 11 575 11 East Kazakhstan 588 10.2 582 10.4 574 10.6 560 10.6 Zhambyl 581 10.1 582 10.4 580 10.7 573 11.0 Kyzylorda 582 10.1 582 10.4 570 10.5 581 11.1 Pavlodar 573 9.9 561 9.9 556 10.2 544 10.2 North Kazakhstan 554 9.4 549 9.6 554 10.1 523 9.7 Aktobe 557 9.5 548 9.6 511 9.1 530 9.9 Karagandy 547 9.3 540 9.4 529 9.5 522 9.7 Astana city 544 9.2 536 9.3 530 9.6 515 9.5 West Kazakhstan 544 9.2 535 9.3 525 9.4 519 9.6 Atyrau 536 9.0 533 9.3 471 8.2 478 8.6 South Kazakhstan 530 8.9 533 9.3 522 9.4 522 9.7 Akmola 538 9.1 529 9.2 528 9.5 511 9.4 Kostanay 549 9.3 523 9.0 516 9.2 494 9.0 Almaty 519 8.7 511 8.8 507 9.0 504 9.3 Mangystau 504 8.3 508 8.7 473 8.2 476 8.6 Source: Author’s calculation on the basis of TIMSS-2015 data for Kazakhstan.

CAP Fellows Paper 224 13 behind Almaty city. This means Figure 5 shows the results of LAYS in regions, providing a broad that in 11 years of school, students calculations for eighth-graders’ range of answers to exploratory there only cover knowledge equal math scores, with learning questions and helping to develop to around 9 years in the schools starting at Grade 0 and Grade -3. an informed hypothesis. of Almaty city. From the map This approach to calculation is above, we can also observe that offered by Filmer et al. to compare Survey questions were clustered Mangystau oblast, technically adjusted learning outcomes for into three major subgroups: level classified as a western region (it is different assumptions on when of awareness; reasons for region- also similar to Atyrau oblast in its the learning starts—either at al underperformance in second- oil-production economy), is geo- Grade 0 of school or 3 years ary education; and what can be graphically closer to the south- before that, when a child goes to done to shift the situation. 57 ern region of Kazakhstan. This kindergarten. The numeraire o Public awareness (is there a fact was previously mentioned by region is Almaty city and the problem?) Alimkhanova in her analysis of correlation between the two the NEET situation in South Ka- measures is 1. Q1. What do you think about the zakhstan.56 quality of secondary education in Predictably, there is also a strong Kazakhstan (as shown by student The results for West Kazakh- correlation between LAYS for achievement, final grades, project stan as a whole are more different subject areas (science work, academic contests)? homogeneous than those and math)—0.93 for eighth-grade for the South. We can observe scores. Six of the experts surveyed con- that both Atyrau and Mangystau sider secondary education to be oblasts are more than two years Expert Survey Results of either low or “below average” behind Almaty city in academ- quality in general. Some of their ic achievement. In their eleven The ultimate goal of the criticisms relate to the perceived years at school, eighth-graders expert survey was to serve overall “elitism” of secondary ed- in Atyrau and Mangystau oblasts as preliminary grounds for a ucation, where a lot of attention is are projected to cover an amount larger study on the reasons for given to catering to talented and/ of learning equal to just 8-8.5 educational underperformance or well-off students rather than to years in an Almaty city school (there is an almost three-year gap Figure 5. LAYS for regions of Kazakhstan in science). For West Kazakh- (eighth-grade math) stan oblast, LAYS varies from 9.4 years in science to 9.6 in math, thus showing an approximate 1.5- year difference from Almaty city. Aktobe oblast is almost 2 years behind Almaty city in science and 1 year behind in math.

Another pattern worth men- tioning can be seen across the Central Kazakhstan re- gion. Karagandy and Akmo- la oblasts, as well as the capital city, Astana (Nur-Sultan), all lag 1.5 years behind Almaty city in achievement.

Source: Author’s calculation on the basis of TIMSS 2015 CAP Fellows Paper 224 data for Kazakhstan. 14 underperforming students. Other experts there is a clear consensus communication skills, etc. In oth- experts pointed to specific prob- that the quality of education can- er words, the range of indicators lems: a disparity in educational not be generalized at national lev- of the quality of education is di- outcomes between urban and ru- el due to the heterogeneity of the rectly or indirectly connected ral schools, regional differences, schools in the system and their with the conceptual platform of and a big gap in achievement be- varying prerequisites. the national system of educa- tween gifted and struggling stu- tion.” dents. Q2. What else do you think can serve as an indicator of the quali- E1: “Skills of working with infor- E5: “Kazakhstan’s education sys- ty of education? mation: find it, analyze it, and tem is like a big field with built-in make conclusions.” ‘quality islands’ resulting, I think, Experts mentioned functional from the education ‘elitization’ literacy (the ability to communi- E9: “The main indicator is the politics realized throughout the cate and work with various types share of graduates enrolled in past decades.” of information, the ability to ap- the world’s top .” ply knowledge to real life) and E13: “…There’s too much focus on university enrollment/competi- E13: “One of the key indicators ‘gifted’ kids, while no work is be- tiveness of graduates as the most of educational quality is reduc- ing done with underperforming important indicators. Equality ing the gap in achievement be- or struggling children.” of opportunity and inclusiveness tween struggling and successful was the second most frequently students.” E11: “There are schools that of- mentioned indicator, while only fer world-class education and two experts mentioned parent, Q4. How well, in your opinion, schools where kids do not learn student, and teacher satisfaction is Kazakhstani society informed even basic skills.” as evidence of the system’s qual- about the achievements or limita- ity. tions of achievements of the sec- Only two respondents consid- ondary education system? What er secondary education to be of E14: “… We need to focus not so do you yourself know about its “good quality” or to have made much on the current numerical key development indicators? significant progress in recent indicators, but rather on qual- years. These respondents also ity indicators, as well as on the Experts held similar views re- mentioned the development of range of indirect impact factors garding the level of social aware- institutions for gifted children in (economic, demographic, so- ness of the achievements and pit- support of their argument, as well ciological) that accompany the falls of the country’s secondary as increased financial support for sphere [of education].” education system. The majority the system as a whole. believe that society is either com- E5: “The indicators mentioned pletely unaware of or insufficient- E14: “I think the quality of sec- here (academic achievements, ly informed about recent devel- ondary education is rising. It participation in school Olym- opments in secondary education. is connected with 100-percent piads and contests) are ‘classic’ school enrollment, good-quality indicators used to make educa- E6: “On average, society does preschool preparation, paren- tional rankings by education- not know anything. [Those on] tal involvement, and increased al systems that still function in Facebook know a set of populist social activism, as well as the the ‘knowledge paradigm.’ If a facts, analysts might know about work of the state government to system of education aims to fa- PISA, etc.” strengthen the quality of educa- cilitate students’ acquisition of tion and the work of Nazarbayev a broad range of competencies, E4: “Society is not sufficiently Intellectual schools58.” the indicators will be broader aware. The secondary education as well—like students’ ability to system in Kazakhstan functions Overall, among the majority of the make decisions, teamwork skills, according to a top-down ap-

CAP Fellows Paper 224 15 proach, which does not always offered the chance to look at the eral. contribute to developing teach- results of three large-scale com- ers’ initiative and their readiness parative studies (TIMSS, PISA, On the other hand, as mentioned to changes. That is why the key PIRLS) and asked whether they by one of the experts, since the indicator of the system’s im- could draw any conclusions from methodology of a particular test provement is their inclusion as that information. is the same for all the regions, experts in the process of develop- it provides a good starting point ing textbooks, curricula, and as- Q5. Is it possible, in your opin- for further analysis, showing that sessment systems, which should ion, to trace the quality of school- particular regions have lower re- contribute to teachers’ profes- ing in specific regions based on sults across all three studies. sional development.” these results? o Reasons for regional under- E3: “The general public in Ka- Overall, there is a high level of performance zakhstan mostly knows about trust in LSAs among the experts the successes and failures of the surveyed. Twelve of 16 experts be- Q3. How important are back- system of secondary education lieve they can be used as reliable ground (contextual) factors in from their own or their commu- sources of information about the determining a child’s academic nity’s personal experience. The quality of secondary education. achievement? (multiple choice) nationwide PR policy on educa- Among the four who answered tion requires improvement.” this question in the negative, two Before addressing specific re- referred to the need to integrate gional underperformance, the E10: “I think the public is not the data into a larger analysis experts were asked to answer a sufficiently informed about the with additional data in terms of multiple-choice question regard- state system of secondary edu- the quantity of observations and ing the influence of contextual cation in Kazakhstan. On the one other indicators such as universi- factors (as opposed to features of hand, this could be due to a lack ty enrollment data. Two experts schooling, happenings in school) of interest on the part of the peo- disregarded the international test on a child’s academic achieve- ple themselves, except those di- results completely, expressing ment. As can be seen from Figure rectly involved in the education- their distrust in the integrity of 6, the majority believe that family al process (teachers, managers, the data and the administration SES and the values system with students and their parents, state of the tests in Kazakhstan in gen- which children are raised play a institutions). On the other hand, much is underpinned by a lack of systematic communication from Figure 6. Role of contextual factors in determining the schools and the state. The academic achievement only key indicator of the devel- opment of secondary education system of which I am aware is the UNT.”59

E12: “They [the public] are com- pletely unaware. We need an effective communications cam- paign, although it is still not in the interests of the state appara- tus to shed light on the real issues at hand.”

After the first segment of five broad questions, experts were Source: Compiled by the author on the basis of expert survey responses.

CAP Fellows Paper 224 16 significant role in their academic E3: “The gap in achievement depend on the urban/rural pop- success. Living standards in their might be due to intraregional ulation ratio, living standards in neighborhood and school infra- characteristics. It is important, the region, ecology, and tradi- structure come second in terms for example, what type of pro- tions.” of their influence on academic duction the region specializes in.” achievement. o Character of change required E5: “South Kazakhstan oblast is Q5. What key factors, in your peculiar due to its high density Q9. What could be done (in both opinion, might influence the of school students from various the short and long term) to shift quality of secondary education in backgrounds; the population has the situation of failure (both re- a region? a more traditional value system gionally and nationwide)? in which education might occupy Q6. In your opinion, can internal a firm position but not be a first It is possible to disaggregate ex- regional characteristics impact priority.” pert opinions here into several underperformance [in secondary categories that emerged during education]? E6: “…However sad or shame- the analysis: a) actor (who should ful it is to acknowledge this fact, author the change); b) character It is possible to identify four [regional underperformance] of change (type of interference re- main factors that the experts might also be connected with the quired); and c) object of influence consider to be the most influ- share of Russian-speaking popu- (what or whom the change should ential in determining region- lation in the region, although this address). The measures suggest- al educational achievement: should change with time.” ed by the experts to reduce in- • Regional SES (including the equality in access to quality ed- share of urban/rural population, E11: “In Mangystau and Atyrau ucation ranged from additional education expenditure, school oblasts, [low educational funding to changing the structure financing, and the poverty level achievement] might be due to the of education system management in the region/neighborhood); outflow of human capital from at both national and regional lev- • Quality of teaching (teachers’ teaching to higher-paid pro- el. qualifications and professional fessions. Moreover, along with development opportunities); South Kazakhstan oblast, these Actor • Language (Kazakh versus regions are more Kazakh-speak- Russian languages of instruction, ing and it is no secret that in- As was to be expected, all experts share of Kazakh-speaking struction in Kazakh is not the link any potential change to the population in the region); and same quality as instruction in central government, represented • Values and traditions of Russian or English, which means by the Ministry of Education and local population (importance of low-quality textbooks and low- other decision-making bodies. education in regional strategic er-quality teachers.” This is a fair perception in a cen- development, traditions and tralized system, where decisions local views regarding the value of E9: “I think the results will cor- involving finance, infrastructure education) relate with the urban/rural and capacity-building are nor- population ratio. The results of mally made only by higher au- E6: “If it is a mining/trading urban students will always be thorities. This is also mostly true or an agriculture-oriented re- higher than those of their rural for region-level activities, espe- gion, it is no place for intellectual peers, so it is not quite right to cially in the field of education— teachers. Moreover, the aver- compare the city of Almaty and regional activities in this sphere age parent would be engaged in Mangystau oblast.” are planned on the basis of the non-intellectual work. It is chal- State Program for Education De- lenging to achieve a high quality E14: “The fact that particular re- velopment, which states national of education in such conditions.” gions are lagging behind might and regional priorities.

CAP Fellows Paper 224 17 Figure 7. Actor, change, and object of change proposed by experts

Source: Compiled by the author on the basis of expert survey responses.

Type of Change prior to enacting new measures. rid of the archaic secondary edu- cation management scheme (dis- The most important and urgent E2: “Support programs for trict/city/oblast authorities), as measure, in experts’ opinion, is weaker regions, targeted sup- they pose a corruption risk and to increase financial support for port for ineffective schools and are of no use.” the education system in general, struggling children.” as well as for the regions. That E13: “On a regional level, [we includes raising teachers’ salaries E3: “Before introducing any should prioritize] 1) developing and targeting financial aid to measures, we need to conduct infrastructure; 2) improving weak schools in disadvantaged deep research into the reasons teacher quality; and 3) adopting neighborhoods. for regional inequality in educa- targeted regional programs with tion. And after the reasons are increased financing. Nation- E2: “… regional support pro- clear, specific regional measures wide, there is a need for a sup- grams, targeted support for weak can be initiated.” port program for disadvantaged schools and struggling children.” children and schools.” E8: “I think we need a specific E1: “Increased financial support, regional policy to reduce the dis- E15: “There is a need for a deep including for poverty alleviation, parity in the quality of education. analysis of the issues underpin- increased salaries for teachers.” Perhaps we need a special salary ning quality of education (impact bonus for teachers in the western factors) and possible solutions.” E12: “Increase per capita fund- region, where the average salary ing, give [schools] autonomy.” [for teachers] is much less than Improvement of teacher quality the region’s average wage.” and status is another step that There is a consensus among the was often mentioned by the ex- majority of experts that measures E9: “I suggest 1) delegating the perts, who are alarmed by the to reduce the achievement gap prerogative of school financing poor quality of future teachers should originate from within the and teacher salaries to the local admitted to training programs region, based on the regional authority level; 2) that the Minis- at universities and the lack of context and issues at hand. Two try of Education should concen- teacher autonomy in Kazakhstan. experts mentioned the need for trate only on coordination of and Some experts view improving the research to understand the reasons methodological support for the status of teachers as a vital part for regions’ poor performance schooling process; and 3) getting of the overall improvement of

CAP Fellows Paper 224 18 the quality of education in the re- ‘Kazakhstani’ model of second- trust in the transparency of their gions. ary schools. It is sufficient to an- decision-making. Finally, there is alyze the models implemented skepticism about the recent re- E10: “[We need] a higher level of in developed countries (like the forms in secondary education, in- teachers (financial stimuli, bet- USA, Finland, , etc.) and cluding trilingual education and ter preparation and more rig- copy one of them. I insist—just the renewed curriculum, with orous selection by pedagogical copy as it is, and do not change some experts suggesting that they programs, focus on attracting even 0.01 percent of its content.” are having a detrimental effect on professionals without teaching the quality of secondary educa- experience but with appropriate Object of Change/Influence tion as a whole. practical expertise.” As for “what has to change,” Discussion E4: “Continuous improvement of there are several areas which, in teachers’ professional excellence, the experts’ opinion, should be “It is certainly true that focused not on their formal qual- targeted to improve the quali- schooling should be easier to ifications but on improvement ty of education both nationwide influence than family, peer, or of their practice; establishing and regionally. As mentioned broader social and economic teaching as a profession rather before, the quality of teachers is trends, but this does not than a semi-profession.” associated with the quality of ed- invalidate their influence.” ucation, which is why there are —Mortimore and Witty (1998, p. E11: “Improving teachers’ sta- a number of recommendations 300) tus, reducing administrative relating to providing support for workload, and motivating them their continuous professional Certain regions of Kazakhstan to engage in continuous (!) ca- development and rethinking the are clearly underperforming pacity-building, including rural enrollment policies of university in secondary education, as teachers.” teacher-training programs. can be seen from the results of the international large-scale It was also suggested to facili- However, qualifications are not assessments and the calculation tate the development of private the only issue. Teachers’ low wag- of learning-adjusted years of schools and to roll back the edu- es were frequently mentioned as schooling (LAYS). cational reforms launched in the the main limiting factor. Teach- past five years, including trilin- ers’ welfare is likewise cited as an According to the LAYS calculation, gual education, which are seen important condition for the holis- by the time they graduate from as possibly too big a burden on tic educational process and is as- school, current fourth- and schools and teachers in particu- sociated with both compensation eighth-graders in Almaty and lar. It is worth mentioning that and the status of the profession. Mangystau oblasts will only have there are certain indications of acquired knowledge equivalent dissatisfaction with the curricu- Experts express their concern to 8.5 years of schooling (out of lum and textbooks—one expert about the top-down school man- an expected 14 in preschool and suggested replacing national agement system, arguing that school). Even bearing in mind textbooks with foreign editions, schools need more autonomy to the possible limitations of such while another believes that the be able to improve. An increase calculations (eg., the extent to whole “Kazakhstani” model of in the number of schools financed which test results represent secondary education is ineffec- through per capita funding is the actual learning happening tive, suggesting that the example also seen as a necessary step to- in the classroom), this seems of developed countries should be ward building a more sustainable unacceptable in a state that is followed instead. school environment. Certain con- determined to provide every cerns and criticisms addressed child with equal educational E7: “[We need to] give up the the work of the regional educa- opportunities and equal quality idea of building a specifically tional authorities, including dis- of education.

CAP Fellows Paper 224 19 Understanding what lies behind aimed at reducing educational for the region, it is not so scary, such drastic differences in inequality across regions based since it is possible to make it at- academic achievement is vital on an understanding of the key tractive to qualified employees/ for planning any interventions impact factors. The two strategic the educated population using to change the situation for the documents outlining educational administrative resources. How- better. As mentioned above, policies and regional development ever, that requires [political] will much of academic achievement is priorities in Kazakhstan, as well as and resources. Nevertheless, in explained by contextual factors, regional education development a stagnating situation, the low some of which were highlighted strategies, hardly take into quality of education means the during the expert survey. account regions’ socioeconomic continuing growth of poverty in context or local conditions. the region and out-migration.” According to respondents, a family’s socioeconomic For example, the State Program E8: “Consequences manifest in status, quality of teachers, a for Education Development lower chances for children to win family’s values, and language of (SPED) outlines strategic a higher education grant and to instruction at school all have a goals for the education system compete in the labor market.” direct influence on the quality of as a whole, aiming mainly at education available to students achieving numerical indicators. E14: “The basic consequence of in Kazakhstan. Yet it appears The State Program for Regional low-quality education in a re- that most of these factors are Development (SPRD),61 in gion is, in my view, out-migra- not accounted for in educational turn, refers to the SPED on all tion—from rural to urban areas planning at regional or national indicators concerning education. and from struggling regions to level, for a variety of reasons: lack Regional education development more well-off ones, like Nur-Sul- of relevant data, the area being strategies, too, are normally tan, Almaty, and Karaganda cit- out of the reach of educational drafted in accordance with and ies. Thus, the gap in education authorities, etc. on the basis of SPED goals and quality is ever increasing.” indicators. Mortimore and Witty’s rhetorical E5: “The main consequence is an question “Should we ignore Under such a unified approach unfair narrowing of the oppor- disadvantage in the hope that to education development, it tunities for sustainable develop- students themselves will find the is not surprising that regions ment available to children and necessary strengths to overcome experience varying effects youth, which in the end results in their problems?”60 sums up the from the government’s well- various negative social phenom- gist of the practical and ethical intentioned national initiatives. ena like poverty, asocial behav- controversies around the issue. While regional inequality in ior, etc.” It is my belief, however, that (at education is continuously least in the case of Kazakhstan) brought to attention in national Besides the legitimate concerns a large share of the contextual or reports and national analyses of about out-migration in the other factors standing in the way TIMSS and PISA results, there south of the country, there of a child’s academic achievement is still no specific strategy or is another issue—the rising can be, if not eliminated, then action plan in place to reduce it. number of NEET youth (those compensated for. To bring this In short, Kazakhstan’s education not in education, employment, about, we need political will and system “does not act as a social or training). The share of NEET a holistic strategy. lift but rather increases social youth in Mangystau oblast inequality.”62 is already 16 percent, and in The main problem with the current South Kazakhstan oblast it is 10 policy on the issue of regional E6: “For a child, [low educa- percent.63 As a rule, this number academic underperformance is tional achievement in a region] is higher in rural areas, indicating that there is virtually none. At means zero opportunities in young people’s lack of interest in least, there is no policy specifically terms of employment and career; entering higher education or the

CAP Fellows Paper 224 20 job market—or their inability to Both the underpopulated oil- Kazakhstan oblast (now divided do so. As Alimkhanova mentions, producing Atyrau and Mangystau into Turkestan oblast and the city NEET youth mostly come from oblasts, with the highest average of Shymkent), for example, had disadvantaged families and salary and share of the country’s the lowest expenses per student demonstrate low academic GDP, as well as the densely in 2015. achievement.64 The implications populated and poor South of such youth disengagement are Kazakhstan region (classified by There is an intrinsic multiple, including increasing Whiteshield Partners and EBRD assumption that all policies prospects of social tension in the in 2015 as “losing”), demonstrate and initiatives should regions and the issue of return on staggering underperformance work the same across all educational investment. in education, a fact that calls for mainstream schools in specific attention. If nothing else, Kazakhstan. However, a The perceived marginalization this could mean that besides drastic difference in academic of born in the Western local educational institutions performance between Kazakh- and Southern regions is evident, themselves, there are other and Russian-language schools perhaps now more than ever. As major factors influencing persists. Thus, the effect of any Koch and White’s study shows, the the effectiveness of regional well-intentioned initiative (like “southerners,” in particular, are secondary education systems trilingual education) is likely to perceived by other Kazakhstanis and that these factors may vary be minimal in Kazakh-language as “aggressive,” “uncivilized,” and from region to region. Among schools, with students receiving “unintelligent,” with a certain these, one might, as experts less benefit in the end. level of social resentment coming have indicated, list language, from the “urban” population of the local values system, and the In 2018, 53 percent of all bigger cities in Central and North socioeconomic situation. schools in Kazakhstan taught Kazakhstan.65 solely in Kazakh, accounting for Language and Values as 40 percent of the total school Moreover, as Koch and White find, Determinants of Academic student population (1.3 million Kazakhstani citizens consider Achievement? students).66 The distribution of southern and western regions Kazakh schools across the country the “least desirable” to live in, There is clear evidence that reflects the density of the native with economic opportunity cited in schools with Kazakh as the language-speaking population. as the most important reason for language of instruction, as well In Mangystau oblast, 86 percent this. There is a common public as in rural schools (and especially of schools teach in Kazakh; perception that people from when these two overlap), students in Kyzylorda oblast, another South Kazakhstan are corrupt. continue to show much lower southern region, the figure is achievement. 89 percent. In the former South In a country of inherent Kazakhstan oblast, there are over controversies, among which However, it may be argued that 1,000 Kazakh schools, comprising language and the opposition strategic planning in regions does 72 percent of all regional schools between traditional modern not control for poorer education and one-fifth of all Kazakh mindsets are some of the most in these schools. Regional schools in the country. Kazakh apparent, the signs of such strategic plans, for example, schools represent a clear majority alarming social segregation might only include generalized in the west too, with such schools cannot be underestimated. It “measures to improve quality of comprising over 65 percent of is imperative to take measures education” and do not stipulate the total in Aktobe, Atyrau, and so that children do not become financial advantages or additional West Kazakhstan oblasts. South hostages of the socio-economic funds for poorer schools or and West Kazakhstan account for or cultural issues of their Kazakh-language schools (which the largest share of the country’s neighborhoods. often overlap). The former South student population studying in Kazakh.

CAP Fellows Paper 224 21 Figure 8. Share of students in Kazakh-language schools (of total regional school population)

Source: IAC, “Education System Statistics in Republic of Kazakhstan,” national report, 2018.

The language-related disparity examples showing that, given the The impact of values, in academic achievement is right resources and curriculum, traditions, and priorities on somewhat unique to Kazakhstan either language can be turned academic achievement is fairly and reflects national history. First to students’ advantage. The understudied and is even harder of all, it is not speakers of the experience of the Nazarbayev to measure and correlate with second or foreign language who Intellectual schools and Kazakh- performance than the language are lagging behind but those of the Turkish Lyceums, which factor. state language—the language of the practice trilingual education, ethnic majority. Also, as has been has proved that it is possible to In Kazakhstan’s educational demonstrated by international create learning environments development programs, the assessments, bilinguals (those in which place of origin or first closest proxy for values is the who speak both Kazakh and language do not impact academic indicator of “satisfaction with Russian at home) score higher achievement, but even become educational reforms and policies/ than their monolingual peers. a student’s strengths. The quality of education.” It is one of graduates of these institutions the target numbers that is set Undoubtedly, the lower quality are the most competitive in the to improve every year.68 This of education in Kazakh schools is country, often being accepted indicator is isolated and probably due in part to the Soviet legacy of to top international universities not a good reflection of all suppressing national languages even before graduation. education policies and the overall and identities, a policy the effects quality of education. Yet it is fair of which are perhaps more vivid The problem is with the country’s to argue that satisfaction can and in Kazakhstan than in any other mainstream schools, which will appear only after the local post-Soviet state. To this day, are typically underfunded in population sees an improvement Kazakh-language university overpopulated areas and struggle in the quality of education in faculties experience a lack of to attract the best university their neighborhoods. Arguably, relevant materials, poorer labs, graduates if—like 75 percent of improvements in Astana (Nur- and lower-quality teaching staff.67 the country’s schools—they are in Sultan), Almaty, or Shymkent rural areas. cities will not prove particularly However, there are several comforting for a parent in newly

CAP Fellows Paper 224 22 established Turkestan oblast69 low value placed on education represented by parents’ income whose child studies in a class with for women by local people, as and education and child’s access forty classmates. well as of the lack of awareness to educational resources, is of basic human rights statutes perhaps the trickiest impact Values play an important role and policies. This also speaks to factor to address from an in understanding the quality of the need for specific programs at educational perspective. As secondary education for girls. schools to educate both boys and arguably the most influential Women are an important part girls about the immorality and factor in predicting a child’s of the country’s human capital. illegality of bride theft, as well as academic success, it therefore Several international studies and to teach girls to raise their voices puts increased responsibility reports highlight the importance and advocate for themselves. on schools in disadvantaged of quality education for women— Ignoring the poor quality of neighborhoods. These schools both in terms of return on education in these regions would have to compensate for poorer investment and in terms of result in, among other things, a educational opportunities and social and cultural capital. Thus, further deterioration of human the family issues that a child making sure that every girl has a rights. often experiences at home. At the chance to get a higher education same time, they struggle to meet or otherwise earn a professional The values system of the the national criteria prescribed qualification enabling her to population is not something that by the unified national education enter the labor market and exists in isolation. As Wilson development program and further develop her abilities is notes, particularly for school- regional strategies. extremely important for the aged children, the value of getting nation. Not only does investment higher education or graduating Thus, we find a situation “where in women bring greater returns from school might depend on schooling may not automatically in the future,70 but women also the benefits associated with transform into human capital transmit their potential and it.73 Thus, we circle back to the because of poor educational values to their children, both socioeconomic situation in which institutions, nor be channeled directly and through the subtle a child finds him- or herself—the into productive use due to lack transmission of social and amount and quality of educational of institutional efficiency in the cultural preferences.71 resources available to him/her economy.”74 and the career opportunities Though almost 100 percent of they associate with secondary or “Quality” of education depends school-age boys and girls are higher education. on a variety of “schooling” factors, enrolled in secondary education including instruction, curriculum, in Kazakhstan, what happens to Disbelief in being able to teachers, school infrastructure, girls after school is an important succeed in life when “living by school governance, and the factor as well. the rules” may lead to conscious quality of management of the disengagement from education. local educational authority. As Both South and West Kazakhstan This is especially likely to be the both literature analysis and are known for a tradition of case in the atmosphere of failure expert surveys show, other factors “bride theft,” which often that inevitably forms around a outside the school—like family happens against a young schoolchild in a disadvantaged SES and priorities—have a major woman’s wishes, often leading area. One example of this is the impact on learning. Therefore, it to tragic consequences.72 These rising NEET phenomenon in seems that there is an inevitable regions also feature frequent South Kazakhstan. policy dichotomy when it comes cases of early marriage. Women to raising a resilient, economically are often seen only as housewives SES of the Region and Family active, moral citizen. and take no part in education or work. This is a direct result of the Socioeconomic status, as While efforts in the education

CAP Fellows Paper 224 23 field might be focused on a important factors in determining academic achievement and his or student’s IQ and EQ, the lack of academic achievement, it is no her future life prospects. In turn, complementary actions or policy surprise that weaker schools the neighborhood and success or implementation failures in other tend not to attract the best failure of a child’s elders might spheres (national economy, social university graduates. Moreover, also impact his/her educational development, health, etc.) puts employment policies are vague choices, affecting his/her utility- educators in a difficult position. at best: because very few maximizing perspective on young people decide to commit education. First of all, there is increased themselves to teaching in rural responsibility. As schools “deal” schools, the latter often employ There is also evidence that, vice with citizens for a longer period TVET () and versa, education is the single of time than any other single school graduates. most important factor influencing institution, society perceives regional economic performance. them as responsible for students’ In such a closed “vicious circle” As the literature indicates, in “holistic development.” This environment, a school might fail line with my survey results, narrative is omnipresent in both to do its own part in forming an there are also other impact official and media publications economically active individual— factors that might significantly in Kazakhstan, where school is making it impossible for it to impact average regional student positioned as a major agent in the compensate for the shortcomings achievement. These factors are “formation of a citizen.” of other spheres. rooted in each region and may vary from one to another. And then there is an unseen Now, if one assumes there is obligation to compensate for a large concentration of such Some of these impact factors are those other institutions and schools in a particular place that unique to the Kazakh context and policies that fail to do their part are failing not only to compensate need to be thoroughly studied. in this very holistic development, for other institutions but even to Understanding the causes of the ones responsible for dealing fulfill the school’s own duties, things is a key condition for with the issues of families with that would have a detrimental planning effective change. Thus, low SES, single mothers, families effect on the whole human it seems so far that it is vital to with children with special capital potential of that region understand a) which impact educational needs, etc., as well and its citizens. Education, and factors are the prerogative of the as the lack of sports and cultural specifically secondary education, main responsible governmental institutions in rural areas. is the foundation of one’s future institutions—the Ministry success. A failure to provide it of Education and regional So when the other policy or seriously undermines not only a authorities—and which are not; economy spheres do not keep up, student’s personal happiness and and b) which of them can be schools in disadvantaged areas professional fulfillment, but also turned into measurable indicators are in a tricky, no-win situation. the country’s future economic and which cannot. That is why we have devoted welfare. That may seem too teachers with high anxiety levels straightforward, yet often we fail Drastic differences in both quality and young teachers not willing to to see this bigger picture—and to of life and quality of education in even start work after graduating remember that even in the case of the regions of Kazakhstan call for from universities.75 a bad school, its being bad is not a targeted, specific approach. If we the primary problem. want to provide equal educational What if the school is a weak opportunities to every child, one itself? As the data show, Conclusion and producing isolated policies and weak schools appear mostly in Recommendations initiatives solely in the education economically disadvantaged, sphere and expecting them to rural, Kazakh-speaking It is evident that the work is not enough. communities. And if the quality socioeconomic background of of teaching is one of the most a family influences a child’s Yet before starting to work on

CAP Fellows Paper 224 24 reducing such a knowledge gap, data would make it possible to for families’ low socioeconomic the Ministry of Education and identify the needs of schools status. Given that exposure to other governmental bodies need that are lagging behind—in learning resources and gadgets to have substantial data, which can terms of teacher training, at home has a strong connection only be obtained through rigorous school infrastructure, and to a child’s learning outcomes, regional research. Moreover, a student body—and target them providing children in remote rural holistic understanding of the specifically at both regional and areas with study rooms with free scope of the problem is needed. national levels. Internet, computers, and libraries As secondary education clearly could contribute to mitigating is and will always be primarily These two are the “larger picture” this effect. It is important to the prerogative of the Ministry measures aimed at getting holistic attract local entrepreneurs and of Education, it is important data and applying it to developing bigger manufacturers as potential to identify exactly what it can specific programs in each region. sponsors. change in the “school territory” • Attracting Bolashak in a particular region, but only In the meantime, certain specific graduates to teaching and as part of a vision, strategy, and steps can already be taken: regional LEAs. Engaging more action plan for the whole region. • Raising standards for “Bolashakers” in civil service has enrollment in teacher training long been discussed in policy and Thus, a two-step policy initiative programs. To provide a high media circles, with the prime is proposed—a nationwide quality of education at schools concern being that they would study of the reasons for regional and to have teachers able to work not be motivated to take low-paid inequality in education, followed within the renewed curriculum jobs. To this end, creating a pool by human capital development framework, standards for of Bolashak graduates for state programs for all regions. The university admissions and service in the regions would be a “human capital” idea rests on the employment need to be raised. good first step toward revitalizing assumption that changing the • Raising salaries for local educational authorities. quality of secondary education teachers and LEA workers. It is also important to create a and improving access to it in Besides the international evidence pathway for graduates of non- any region should be a priority that teachers’ salaries correlate pedagogical faculties to teach in of not one but all stakeholder positively with teaching quality schools, perhaps by introducing institutions and government and academic achievement, a PGCE-like program. Providing bodies. this is also important to attract the best university graduates the best graduates to teaching. with competitive wages is a vital Convening a group of qualified Implementing these two steps aspect of retaining them in a specialists (researchers, simultaneously is a precondition given profession. To do this, we data scientists, economists, for the success of both. could draw on the worldwide sociologists) unaffiliated with • Applying positive experience of Teach for All: its any of the local offices is vital for discrimination in terms of Russian analogue, for example, the success of the research stage. financial support for schools invited large companies to Surely, international experience in rural areas. While research sponsor program graduates’ on turnover initiatives for poorly- will highlight the impact factors salaries. performing regions should be for academic achievement, • Inviting more private analyzed closely. the disadvantaged reality of companies and sponsors to rural schools is plain as day. invest in/open private schools The theoretical goal of this Certain measures can already and sponsor public schools. research would be to understand be taken, including increasing Offering inducements like lighter the impact of every possible financing and developing school regulations for private schools in variable (including SES, language infrastructure. South Kazakhstan region could and local values, and any other • Opening public study boost entrepreneurial interest in “invisible” factors) on academic facilities in disadvantaged such activities. achievement. In practice, the neighborhoods to compensate

CAP Fellows Paper 224 25 Appendix. LAYS calculation

Step 1—calculating LAYS on the assumption that learning starts at school

To calculate region-level learning adjusted years of schooling (LAYS), I follow Filmer et al.’s formula:

LAYSc = Sc * R n/c

Sc is “the average years of schooling acquired by relevant cohort of the population,” and R n/c represents “a measure of learning for a relevant cohort of students in country c, relative to numeraire (benchmarking country)”—in other words, learning productivity.76 It is calculated as the ratio of average learning happening per year in respective countries: R n/c = L c/n.

I adjust this formula to calculate region-level LAYS for Kazakhstan. I use mean years of schooling for Kazakhstan as calculated by Barro and Lee (originally used for LAYS measure),77 and I take Almaty city as a benchmarking region (Filmer et al. used Singapore’s scores for international-level analysis).

Thus, expected years of schooling for Kazakhstan (Sc) is set at 14 (11 years of school plus 3 years of preschool). However, I also performed calculations on the assumption that learning starts at school (grade 1),78 thus getting the LAYS results for 11 years of schooling instead of 14.

TIMSS 2015 scores for eighth grade are used to calculate region’s learning per year (Lc). For example, if Almaty’s score in eighth-grade math is 575 and prior years of schooling equal 8, then its average learning per year is 71.2. To compare, Mangystau oblast’s score is 476, which gives us 59.5 as its average learning per year. Thus, R n/c for Mangystau oblast would be 59.5/71.2=0.83, compared to 1 for Almaty city, which is the benchmarking region. This allows us to calculate LAYS for Mangystau oblast according to the formula above as equal to 9.1 years (see Table 7).

This means that by graduation, students in Mangystau oblast will have covered approximately 9.1 years of learning out of the expected 11.

These calculations are then modified to account for the years that take place prior to school.

Step 2—modifying LAYS on the assumption that learning starts 3 years prior to school

In their explanation of the LAYS measure, Filmer et al. pay attention to the question of when learning actually starts and how this might impact the LAYS calculation.79 Table 5 shows the LAYS data I got on the assumption that learning starts at first grade (i.e., at school).

However, as the authors mention, “every child acquires some language, mathematical concepts, reasoning skills and socioemotional skills before arriving at school.”80 To adjust the formula for the years of learning taking place prior to school, Lc is now calculated as the ration of test score (T) to the sum of years of schooling prior to assessment and years of learning prior to school (3+8).

To come back to the example of Mangystau oblast, its Lc will now be equal to 43.2 (score of 476 divided by sum of 3+8). Almaty city’s learning per year will be equal to 52.3. Thus, R n/c (learning productivity) for Mangystau oblast will now be 0.82.

Therefore, the modified formula, accounting for 3 years of learning prior to school, will be:81 LAYSc=[Sc*R n/c]-[Y^p*(1-R n/c]

CAP Fellows Paper 224 26 It is clear that using this second assumption (as I did in my work) will result in lower LAYS scores. I think that this approach is truer to life than the first assumption that learning only starts at school.

Table 7. (LAYS) for regions of Kazakhstan (learning starts at school)

TIMSS TIMSS TIMSS TIMSS score score score score 4th grade, LAYS 4th grade, LAYS 8th grade, LAYS 8th grade, LAYS science math science math Almaty city 623 11 608 11 591 11 575 11 East Kazakhstan 588 10.4 582 10.5 574 10.7 560 10.7 oblast Zhambyl oblast 581 10.3 582 10.5 580 10.8 573 11.0 Kyzylorda oblast 582 10.3 582 10.5 570 10.6 581 11.1 Pavlodar oblast 573 10.1 561 10.1 556 10.3 544 10.4 North 554 9.8 549 9.9 554 10.3 523 10.0 Kazakhstan Aktobe oblast 557 9.8 548 9.9 511 9.5 530 10.1 Karagandy oblast 547 9.7 540 9.8 529 9.8 522 10.0 Astana city 544 9.6 536 9.7 530 9.9 515 9.9 West Kazakhstan 544 9.6 535 9.7 525 9.8 519 9.9 oblast Atyrau oblast 536 9.5 533 9.6 471 8.8 478 9.1 South Kazakhstan 530 9.4 533 9.6 522 9.7 522 10.0 oblast Akmola oblast 538 9.5 529 9.6 528 9.8 511 9.8 Kostanay oblast 549 9.7 523 9.5 516 9.6 494 9.5 Almaty oblast 519 9.2 511 9.2 507 9.4 504 9.6 Mangystau 504 8.9 508 9.2 473 8.8 476 9.1 oblast Source: Author’s calculation on the basis of TIMSS-2015 data for Kazakhstan

Correlation between different LAYS calculations

Correlation between two LAYS measures for eighth-grade math (learning starts 3 years prior to school and learning starts at school, in first grade) is 1.

Correlation between two LAYS measures for TIMSS 2015 eighth-grade science and math scores is 0.93.

CAP Fellows Paper 224 27 Notes (2000-2017),” World Bank Group, Competitiveness Report 2018,” Washington, D.C., 2018, http:// World Economic Forum, Geneva, 1 I. V. S. Mullis, M.O. Martin, P. Foy, documents.worldbank.org/curated/ http://www3.weforum.org/docs/ and M. Hooper, “TIMSS 2015 Inter- en/390321538076747773/Global- GCR2018/05FullReport/TheGlobal national Results in Mathematics,” Dataset-on-Education-Quality-A- CompetitivenessReport2018.pdf. 2016, accessed July 6, 2019, http:// Review-and-Update-2000-2017. 19 Hanushek and Woessmann, timssandpirls.bc.edu/timss2015/ 10 Elena Pelinescu, “The Impact “The High Cost of Low Educational international-results/timss-2015/ of Human Capital on Economic Performance,” 12. mathematics/student-achievement/. Growth,” Procedia Economics and 20 Robert J. Barro and Jong-Wha 2 AAP, “Education Minister Finance 22 (2015): 184-190. doi: Lee, “A New Data Set of Educational Embarrassed by Australia’s 10.1016/s2212-5671(15)00258-0. Attainment in the World, 1950– Declining Maths and Science 11 Robert J. Barro, “Education as a 2010,” Journal of Development Results,” 9News, November 30, Determinant of Economic Growth,” Economics 104 (2013): 184-198. 2016, https://www.9news.com. in Education in the Twenty-First doi: 10.1016/j.jdeveco.2012.10.001; au/national/australian-students- Century, ed. E. Lazear (Stanford, Deon P. Filmer, F. Halsey Rogers, still-falling-behind/b67b2f62-c07f- CA: Hoover Institution Press, Noam Angrist, and Shwetlana 402d-b96f-b356971cc2fe. 2002), 9-24, http://media. Sabarwal, “Learning-Adjusted Years 3 Information-Analytic hoover.org/sites/default/files/ of Schooling (LAYS): Defining A Centre, “Osnovnye rezultaty documents/0817928928_9.pdf. New Macro Measure of Education mezhdunarodnogo issledovaniia 12 Ibid., 17. (English),” World Bank Group, PISA-2015: Natsional’nyi otchet,” 13 Eric Hanushek and Ludger Washington, D.C., 2018, http:// Astana, 2017, http://iac.kz/sites/ Woessmann, “The High Cost of documents.worldbank.org/curated/ default/files/nac_otchet_pisa- Low Educational Performance: en/243261538075151093/Learning- 2015_final.pdf. The Long-Run Economic Impact of Adjusted-Years-of-Schooling-LAYS- 4 In 2019, South Kazakhstan oblast Improving PISA Outcomes,” OECD, Defining-A-New-Macro-Measure- was split into Turkestan oblast and Paris, 2010, https://www.oecd.org/ of-Education. city of Shymkent, which gained pisa/44417824.pdf. 21 George Psacharopoulos and “republican significance” status. 14 Richard Blundell, Lorraine Harry Anthony Patrinos, “Returns Since data for 2015 assessments is Dearden, Costas Meghir, and Barbara to Investment in Education: A available only for South Kazakhstan Sianesi, “Human Capital Investment: Decennial Review of the Global as a whole, I refer to this old The Returns from Education and Literature (English),” World Bank territorial division where applicable. Training to the Individual, the Firm Group, Washington D.C., 2018, 5 Robert E. Lucas, Jr., “On and the Economy,” Fiscal Studies http://documents.worldbank.org/ the Mechanics of Economic 20, no. 1 (1999): 1-23. doi: 10.1111/ curated/en/442521523465644318/ Development,” Journal of Monetary j.1475-5890.1999.tb00001.x. Returns-to-investment-in- Economics 22, no. 1 (1988): 3-42. 15 Aart C. Kraay, “Methodology education-a-decennial-review-of- doi: 10.1016/0304-3932(88)90168- for World Bank Human Capital the-global-literature. 7. Index (English),” World Bank 22 See, for example, the decennial 6 World Economic Forum, “The Group, Washington, D.C., 2018, review of the literature on the returns Global Human Capital Report http://documents.worldbank.org/ to investments by Psacharopoulos 2017,” 2017, http://www3.weforum. curated/en/300071537907028892/ and Patrinos, “Returns to Investment org/docs/WEF_Global_Human_ Methodology-for-a-World-Bank- in Education.” Capital_Report_2017.pdf. Human-Capital-Index. 23 Blundell et al, “Human Capital 7 Lucas, “On the Mechanics of 16 Noam Angrist, Simeon Djankov, Investment,” 4. Economic Development.” Pinelopi Koujianou Goldberg, and 24 Nicola Gennaioli, Rafael La Porta, 8 World Bank Group, “Human Harry Anthony Patrinos, “Measuring Florencio Lopez-de-Silanes, and Capital Index. Kazakhstan (2-Page Human Capital (English),” World Andrei Shleifer, “Human Capital Brief),” Washington D.C., 2018, Bank Group, Washington, D.C., 2019, and Regional Development,” The https://databank.worldbank.org/ http://documents.worldbank.org/ Quarterly Journal of Economics data/download/hci/HCI_2pager_ curated/en/540801550153933986/ 128, no. 1 (2013): 105-164. doi: KAZ.pdf. Measuring-Human-Capital. doi:10.1093/qje/qjs050. 9 Harry Anthony Patrinos and Noam 17 World Economic Forum, “The 25 Ibid., 152. Angrist, “Global Dataset on Education Global Human Capital Report 2017.” 26 Psacharopoulos and Patrinos, Quality : A Review and Update 18 Klaus Schwab, “The Global “Returns to Investment in

CAP Fellows Paper 224 28 Filmer et al., “Learning-Adjusted 37 Kathryn Wilson, “The Education.” Years of Schooling (LAYS).” 27 Determinants of Educational Blundell et al., “Human Capital 54 Information-Analytic Centre, Attainment: Modeling and Investment.” “Natsional’nyi doklad o sostoianii 28 Estimating the Human Capital Psacharopoulos and Patrinos, i razvitii sistemy obrazovaniia Model and Education Production “Returns to Investment in Respubliki Kazakhstan (za gody Functions,” Southern Economic Education.” Nezavistimosti Kazakhstana).” 29 Journal 67, no. 3 (2001): 518. Angrist et al., “Measuring Human 55 World Bank Group, “Kazakhstan” 38 Chandra Muller and David Capital (English).” country brief, 2018, accessed 30 Kerbow, “Parent Involvement in the Hanushek and Woessmann, July 22, 2019, https://databank. Home, School, and Community,” “The High Cost of Low Educational worldbank.org/data/download/hci/ in Parents, Their Children, and Performance.” HCI_2pager_KAZ.pdf. 31 Schools, 1st ed., ed. J. Coleman Barro, “Education as a 56 Dinara Alimkhanova, Determinant of Economic Growth.” (New York: Routledge, 2018), 192. 39 “Understanding the Rising 32 Wilson, “The Determinants of Jessica Harding, Pamela Morris, NEET Phenomenon in Southern Educational Attainment,” 545. and Diana Hughes, “The Relationship Kazakhstan,” CAP Paper 206, 40 Edgerton et al., “Back to the between Maternal Education and George Washington University, Basics,” 877. Children’s Academic Outcomes: A Washington, DC, June 2018, 41 Ibid., 880. Theoretical Framework,” Journal https://centralasiaprogram.org/ 42 Melaku Tesfa Tesema and Johan of Marriage and Family 77, no. archives/12460. Braeken, “Regional Inequalities 57 Filmer et al., “Learning-Adjusted 1 (2015): 60-76. doi: 10.1111/ and Gender Differences in Years of Schooling (LAYS).” jomf.12156; Julia Crede, Linda Academic Achievement as 58 The Nazarbayev Intellectual Wirthwein, Nele McElvany, and a Function of Educational schools (NIS) are a network of Ricarda Steinmayr, “Adolescents’ Opportunities: Evidence from 20 schools for gifted students Academic Achievement and Ethiopia,” International Journal of established in 2008. They were also Educational Development (2018): Life Satisfaction: The Role of a main developer of the renewed 51-59, 52. Parents’ Education,” Frontiers national curriculum. 43 Gennanioli et al., “Human Capital in Psychology 6 (2015). doi: 59 Unified National Testing, a and Regional Development.” 10.3389/fpsyg.2015.00052. standardized test that school 44 Ibid., 129. 33 Susana Claroa, David graduates must pass as a final 45 Ibid., 152. Paunesku, and Carol Dweck, examination. 46 Oblast – administrative and “Growth Mindset Tempers the 60 Mortimore, The Road to Effects of Poverty on Academic territorial division of Kazakhstan. Improvement. Achievement,” Proceedings of the 47 PISA assesses the functional 61 Ministry of Economy of National Academy of Sciences 113, literacy of 15-year-olds. Kazakhstan, “Programma razvitiia no. 31 (2016): 8664-8668. doi: 48 Angrist et al., “Measuring Human regionov do 2020 goda,” http:// 10.1073/pnas.1608207113; Capital (English).” economy.gov.kz/ru/pages/ Peter Mortimore, The Road to 49 The international report on programma-razvitiya-regionov-do- Improvement: Reflections on School PISA 2015 results only includes 2020-goda-1. Effectiveness (Abingdon: Swets and Kazakhstan’s results for closed- 62 S. Irsaliyev, “Kazakhstan in Zeitlinger, 1998), 299-315. ended questions. TIMSS and PISA 2015: Results and 50 34 Jason D. Edgerton, Tracey Peter, I am analyzing data for 16 regions Impact Factors” (presentation at the and Lance W. Roberts, “Back to the instead of 17 because TIMSS 2015 Collegial Meeting of the Ministry Basics: Socio-Economic, Gender, was administered before the division of Education and Science, 2017), and Regional Disparities in Canada’s of South Kazakhstan. Thus, separate accessed July 22, 2019, http://iac. Educational System,” Canadian data for academic achievement of kz/ru/events/kazahstan-v-timss- Journal of Education 31, no. 4 Shymkent city and Turkestan oblast, i-pisa-2015-rezultaty-i-faktory- (2008): 861-888. as well as data on their economic vliyaniya-prezentaciya. 35 Blundell et al., “Human Capital performance in 2015, are not 63 Alimkhanova, “The Rising NEET Investment.” available at present. Phenomenon.” 51 36 Harding, Morris, and Hughes, Filmer et al., “Learning-Adjusted 64 Ibid. “The Relationship between Maternal Years of Schooling (LAYS).” 65 Natalie Koch and Kristopher 52 Education and Children’s Academic Ibid, p. 3. White, “Cowboys, Gangsters, and 53 Outcomes.” For a detailed explanation, see Rural Bumpkins: Constructing the

CAP Fellows Paper 224 29 YouTube video, 7:19, posted by https://www.ebrd.com/news/2015/ ‘Other’ in Kazakhstan’s ‘Texas,’” “Pervyi kanal Evraziia,” November new-study-urges-diversification-of- in Kazakhstan in the Making: 19, 2017, https://www.youtube. kazakhstans-economy.html. Legitimacy, Symbols, and Social com/watch?v=hj1w9jIMZSs; 85 Information-Analytic Centre, Changes, ed. Marlene Laruelle “21-letnii paren’ ukral “Natsional’nyi doklad o sostoianii (Lanham, MD: Lexington Books, nesovershennoletniuiu devushku v i razvitii sistemy obrazovaniia 2016), 181-207. Mangistauskoi oblasti,” Karavan. Respubliki Kazakhstan po itogam 66 IAC, “Education System Statistics kz, June 15, 2018, https://www. 2015 goda,” Astana, 2016, http:// in Republic of Kazakhstan,” national caravan.kz/news/21letnijj-paren- iac.kz/sites/default/files/nd-2016_ report (2018). ukral-nesovershennoletnyuyu- po_itogam_2015_goda_kratkaya_ 67 Beles Centre for Strategy and devushku-v-mangistauskojj- versiya_rus.yaz._iac_zashchita. Analysis, “Teachers of Kazakhstan: oblasti-455015/. pdf; Information-Analytic Centre, Why Young People Choose This 73 Wilson, “The Determinants of “Natsional’nyi doklad o sostoianii Profession and Why They Stay in It,” Educational Attainment,” i razvitii sistemy obrazovaniia 2019, https://beles-cas.kz/images/ 74 Humna Ahsan and M. Emranul Respubliki Kazakhstan (za gody teacher.pdf. Haque, “Threshold Effects of Nezavistimosti Kazakhstana),” 68 Decree of the President of Human Capital: Schooling and Astana, 2017, http://iac.kz/sites/ the Republic of Kazakhstan of Economic Growth,” Economics default/files/nacdok-2017_ot_kgk_ March 1, 2016, “Ob utverzhdenii Letters 156 (2017): 48-52. final_09.08.2017_10.00-ilovepdf- Gosudarstvennoi programmy 75 See, for example, Beles Centre for compressed.pdf; Information- razvitiia obrazovaniia i nauki Strategy and Analysis, “Teachers of Analytic Centre, “Natsional’nyi Respubliki Kazakhstan na Kazakhstan.” doklad o sostoianii i razvitii sistemy 2016-2019 gody,” 2016, 76 Filmer et al., “Learning-Adjusted obrazovaniia Respubliki Kazakhstan https://tengrinews.kz/zakon/ Years of Schooling (LAYS),” 7. po itogam 2017 goda,” Astana, 2018, prezident_respubliki_kazahstan/ 77 Barro and Lee, “A New Data Set of http://iac.kz/sites/default/files/ konstitutsionnyiy_stroy_i_osnovyi_ Educational Attainment.” nacionalnyy_doklad_za_2017_ gosudarstvennogo_upravleniya/id- 78 TIMSS 2015 was administered god_s_oblozhkami_dlya_sayta.pdf. U1600000205/. in Kazakh schools at the end of 86 Statistics Committee of 69 In 2018, South Kazakhstan spring (the school year ends on Kazakhstan, “Regions of Kazakhstan oblast was divided into the city May 25). Thus, at that stage, a child in 2017,” Astana, 2018, http://stat. of Shymkent, which now has the had completed 8 years of school gov.kz. status of “republican importance,” education. 87 In 2017, US$1 was equal (on and Turkestan oblast, which has its 79 Filmer et al., “Learning-Adjusted average) to 332 KZT (see https:// administrative center in the city of Years of Schooling (LAYS).” corner.kz/post/usdkzt-za-dekabr-i- Turkestan. 80 Ibid., 11. ves-2017-god). 70 Richard Blundell, Lorraine 81 See Filmer et al., “Learning- 88 Information-Analytic Centre, Dearden, Alissa Goodman, and Adjusted Years of Schooling (LAYS),” “Analiz rezul’tatov vneshnei Howard Reed, “The Returns to 14. otsenki uchebnykh dostizhenii Higher Education in Britain: 82 Ben Durbin, “One-Dimensional uchashchikhsia 9 klassov (VOUD- Evidence From a British Interpretations of TIMSS and PISA 2015),” Astana, 2015, http://iac. Cohort,” The Economic Journal Shouldn’t Distract from Their kz/ru/analytics/analiz-rezultatov- 110, no. 461 (2000): F82-F99. doi: Genuine Value,” Times Educational vneshney-ocenki-uchebnyh- 10.1111/1468-0297.00508. Supplement, December 8, 2016, dostizheniy-uchashchihsya-9- 71 Harding, Morris, and Hughes, https://www.tes.com/news/one- klassov-voud-2015. ““The Relationship between dimensional-interpretations-timss- 89 Information-Analytic Centre, Maternal Education and Children’s and-pisa-shouldnt-distract-their- “Natsional’nyi doklad o sostoianii Academic Outcomes.” genuine-value. i razvitii sistemy obrazovaniia 72 “Krazhi nevest v Kazakhstane 83 The authors of the Whiteshield Respubliki Kazakhstan po itogam v poslednee vremia priobreli Partners study did not include 2015 goda.” nebyvalyi razmakh,” Zakon.kz, Mangystau and Kyzylorda oblasts November 5, 2012, https://www. due to “lack of diversity.” zakon.kz/4522455-krazhi-nevest- 84 Whiteshield Partners, v-kazakhstane-v-poslednee.html; “Diversification of Kazakhstan’s “Krazha nevest v Kazakhstane— Economy: A Capability-Based komu tragediia, komu schast’e,” Approach,” EBRD, Astana, 2015,

CAP Fellows Paper 224 30