ULVIYYA MIKAYILOVA AND ELMINA KAZIMZADE

11. FACING THE RESEARCH CHALLENGES

Lessons Learned from Monitoring Studies on Reforms in

INTRODUCTION

In 1999, Azerbaijan initiated an Education Sector Reform Project aimed to reform primary and secondary public schooling in the country. The reform was led by the Ministry of Education with co-funding and technical assistance of the . In order to evaluate this major education reform process, three monitoring studies were conducted by two local NGOs – the Center for Innovations in Education and the “Sigma” Research Center for Development and International Collaboration – under Second Education Sector Development Project. These studies have a special importance in the context of Azerbaijan’s education reform because they have the potential to inform policy makers and education officials about education system actors’ views of and attitudes about the reform process implementation. Importantly, these studies have also introduced a newly emerging culture of national policy- making process based on research evidence and a practice of “providing … teachers and other education system participants… with opportunities for voice” (Meng, Panz, & Yangx, 2014, p. 2). However, these monitoring studies have also revealed some challenges of conducting quantitative evaluation surveys in Azerbaijan. One particular challenge was related to the evidence of response bias to sensitive questions about teachers’ attitudes towards the overall education reform, including curriculum reform, support provided by their own schools, assessment of their professional preparedness to innovations, and actual implementation of new curriculum at their classroom level. Usually, socially desirable responses may have different manifestations, ranging from “reporting incorrect information, omitting information or altering the magnitude of the reported information” (Fadnes, Taube, & Tylleskär, 2008, p. 5). What respondents say “may not be true or not entirely true,” showing “a consistent distortion from reality” (Johnson et al., 2002, p. 193). Such bias “can lead to conclusions that are systematically different from the truth,” leading to “false associations or failure to identify true relationships” (Fadnes et al., 2008, p. 1). In light of these challenges, one could wonder what kind of reality the monitoring studies project to policy makers. What kind of conclusions may national policy makers come to when using findings of the monitoring studies? To what extent were the monitoring studies able

I. Silova et al. (Eds.), Reimagining Utopias, 181–198. © 2017 Sense Publishers. All rights reserved. U. MIKAYILOVA & E. KAZIMZADE to reveal “the truth” about ongoing curricular reform? And what factors may have influenced teachers in Azerbaijan to provide false or not entirely true responses? Another challenge was a stable non-response rate to questions related to the daily teaching processes of participating teachers. There are various factors which may prompt respondents to say “no” in the survey questions. For instance, the participants may choose not to answer a question when they “lack the necessary information and/or experience with which to form an attitude” (Krosnick, 2002, p. 89). They may also prefer not to answer some questions to avoid constructing a socially unattractive image of themselves. A fear that information provided by respondents will be revealed publicly could cause social desirability bias in surveys (Fadnes et al., 2008). Teachers who participated in the monitoring studies presumably had knowledge, interest, and information about the new curriculum and understood the importance of those studies, yet many chose not to respond to some questions. Why did teachers decide not to respond to the survey questions related to their everyday teaching practices? Can teachers as state employees provide a reliable data about their work in the context of current political and ethical conditions in Azerbaijan? In the anonymously administered survey, why do teachers in Azerbaijan fear that their responses would be revealed publicly? This paper explores possible drivers of social desirability bias in Azerbaijan’s school context in order to understand and explain what causes response bias and high non-response rate among teacher respondents in the national context. Following the analysis of methodological challenges faced during conducting qualitative research in a post-totalitarian context, this paper examines methodological challenges researchers face in monitoring curriculum reform in Azerbaijan.

THE REFORM CONTEXT: THE CHALLENGES OF CONDUCTING EDUCATION MONITORING STUİDES

Following the breakdown of the in 1991, Azerbaijan has regained its independence for the second time.1 As the country began to revive from political chaos, economic crisis, and social upheaval in the middle of the 1990s, the Government has articulated its development priorities. According to the national development concept “Azerbaijan 2020: Look into the Future”, these priorities included “increasing the effectiveness and competitiveness of the ,” “securing the progress based on innovations,” “laying the foundations of transition from a traditional economy to a ‘knowledge-based economy’,” and “development of human capital… which leads to drastic changes in the education system (Presidential Office of the Republic of Azerbaijan, 2012, p. 7). In 1999, the Government approved the “Education Reforms Program of the Republic of Azerbaijan,” which stressed the need for “adaptation of Azerbaijani education system to standards of the world education system” (State Commission on Education Reforms of the Republic of Azerbaijan, p. 2). This goal was further specified as integration of “Azerbaijani education into the European education system” as reflected in the national education

182