Revista De Educación Nº 386 OCTOBER-DECEMBER 2019 Revista De Educación Nº 386 OCTOBER-DECEMBER 2019 Nº 386 October-December 2019 Quarterly Journal Starting Year: 1952

revista de eDUCACIÓN Nº 386 OCTOBER-DECEMBER 2019 revista de eDUCACIÓN Nº 386 OCTOBER-DECEMBER 2019 Nº 386 October-December 2019 Quarterly Journal Starting year: 1952

MINISTERIO DE EDUCACIÓN Y FORMACIÓN PROFESIONAL SECRETARÍA DE ESTADO DE EDUCACIÓN Y FORMACIÓN PROFESIONAL

Instituto Nacional de Evaluación Educativa Paseo del Prado, 28, 4.ª planta 28014 Madrid España

Edita © SECRETARÍA GENERAL TÉCNICA Subdirección General de Atención al Ciudadano, Documentación y Publicaciones

Catálogo de publicaciones del Ministerio: sede.educacion.gob.es Catálogo general de publicaciones oficiales: publicacionesoficiales.boe.es

Edición: 2019 NIPO línea: 847-19-002-9 NIPO ibd: 847-19-001-3 ISSN línea: 1988-592X 0034-8082 ISSN papel: 0034-8082 Depósito Legal: M.57/1958

Diseño de la portada: Dinarte S.L. Maqueta: Solana e hijos, Artes Gráficas S.A.U. MANAGING BOARD EDITORIAL TEAM

CHAIR Editor-in-chief: Jorge Mañana Rodríguez Alejandro Tiana Ferrer Secretario de Estado de Educación y Formación Profesional Collaborators: Ruth Martín Escanilla y Óscar Urra Ríos

MEMBERS Fernando Gurrea Casamayor Subsecretario de Educación y Formación Profesional SCIENTIFIC ADVISERS Consuelo Vélaz de Medrano Ureta Directora General de Evaluación y Cooperación Territorial

Clara Sanz López International Directora General de Formación Profesional

Diego Fernández Alberdi Aaron Benavot (State University of New York, SUNY-Albany); Abdeljalil Director General de Planificación y Gestión Educativa Akkari (Universidad de Ginebra); Mark Bray (University of Hong Kong); José Joaquín Brunner (Universidad Diego Portales, Chile); Dirk Hastedt Liborio López García Secretario General Técnico (Executive Director, International Association for the Evaluation of Educational Achievement, IEA); Felipe Martínez Rizo (Consejero Técnico Carmen Tovar Sánchez del INEE, México); Marie-Hélène Doumet (INES Programme, OCDE); Directora del Instituto Nacional de Evaluación Educativa Andreas Schleicher (Director, Directorate for Education and Skills, OCDE). Amparo Barbolla Granda Subdirectora General de Atención al Ciudadano, Documentación y Publicaciones

Nuria Manzano Soto National Directora del Centro Nacional de Innovación e Investigación Educativa

Montserrat Grañeras Pastrana Teresa Aguado Odina (Universidad Nacional de Educación a Distancia); Subdirectora General de Ordenación Académica. Margarita Bartolomé (Universitat de Barcelona); Antonio Bolívar Responsable de la Unidad de Igualdad (Universidad de Granada); Josefina Cambra Giné (Colegio de Doctores y Licenciados); Anna Camps i Mundó (Universitat Autònoma de Barcelona); EDITORIAL BOARD César Coll Salvador (Universitat de Barcelona); Agustín Dosil Maceira (Universidad LiberQuaré); Gerardo Echeita Sarrionandia (Universidad CHAIR Autónoma de Madrid); Juan Manuel Escudero Muñoz (Universidad de Carmen Tovar Sánchez Murcia); Mariano Fernández Enguita (Universidad Complutense de Madrid); MANAGING EDITOR Joaquín Gairín Sallán (Universitat Autònoma de Barcelona); José Luis García José Luis Gaviria Soto Garrido (Universidad Nacional de Educación a Distancia); Daniel Gil Pérez ASSISTANT EDITOR (Universitat. de València); Fuensanta Hernández Pina (Universidad de David Reyero García Murcia); Carmen Labrador Herraiz (Universidad Complutense de Madrid); Miguel López Melero (Universidad de Málaga); Elena Martín Ortega MEMBERS Antonio Cabrales Goitia (University College London); Caterina Casalmiglia (Universidad Autónoma de Madrid); Rosario Martínez Arias (Universidad. (Universitat Autònoma de Barcelona); Antonio Lafuente García (Consejo Complutense de Madrid); Inés Miret (Neturity S.L,, Madrid); Juan Superior de Investigaciones Científicas); Leoncio López-Ocón Cabrera Ignacio Pozo (Universidad Autónoma de Madrid); Joaquim Prats Cuevas (Consejo Superior de Investigaciones Científicas); Carlos Marcelo García (Universidad de Sevilla); Miquel Martínez Martín (Universitat de (Universitat de Barcelona); Manuel de Puelles (Universidad Nacional de Barcelona); Francisco Michavila Pitarch; (Universidad Politécnica de Madrid); Educación a Distancia); Tomás Recio Muñiz (Universidad de Cantabria); Luis Juan Manuel Moreno Olmedilla (Banco Mundial); Clara Eugenia Núñez Rico Romero (Universidad de Granada); Juana M.ª Sancho Gil (Universitat. (Universidad Nacional de Educación a Distancia); Lucrecia Santibáñez (Claremont Graduate University); Denise Vaillant (Universidad ORT, de Barcelona); Mercedes Vico Monteoliva (Universidad de Málaga); Antonio Uruguay); Pablo Zoido (Banco Interamericano de Desarrollo). Viñao Frago (Universidad de Murcia).

Presentation

Revista de Educación is a scientific journal published by the Ministerio de Educación, Cultura y Deporte. Founded in 1940, and since 1952 called Revista de Educación, it has been a privileged witness of the development of education in the last decades, and an acknowledged means for the dissemination of education research and innovation, both from a national and international perspectives. It is currently assigned to the Instituto Nacional de Evaluación Educativa within the Dirección General de Evaluación y Cooperación Territorial and it is published by the Subdirección General de Documentación y Publicaciones of the Ministerio de Educación, Cultura y Deporte.

Each year we publish four issues. Starting next issue (No. 361), the magazine will have three sections: Research, Essays and Education Experiences, all of them submitted to referees. In the first issue of the year there is also an indexof bibliography, and in the second number a report with statistic information about the journal process of this period and the impact factors, as well as a list of our external advisors.

From 2006 to the second number of 2012 (May-August 358), Revista de Educación was published in a double format, paper and electronic. The paper edition included all the articles in the especial section, the abstracts of articles pertaining to the rest of sections, and an index of reviewed and received books. The electronic edition contains all articles and reviews of each issue, and it is available through this web page (www.mecd.gob.es/revista-de-educacion/), where it is possible to find more interesting information about the journal. From the 358 number Revista de Educación becomes exclusively an online publication.

Revista de Educación assesses, selects and publishes studies framed in well established lines of research, mainly: methodologies of education investigation and assessment; analysis of education systems and public policies; evolution and history of contemporary education systems; education reforms and innovations; quality and equity in education; curriculum; didactics; school organization and management; attention to diversity and inclusive education; educational guidance and tutorship; teacher selection, training and professional development; international cooperation for the development of education.

5 Revista de Educación is available through the following data bases:

■ National databases: isoc, beg (gencat), psicodoc, dialnet, y redined (Red de Bases de Datos de Información Educativa).

■ International databases: Social Sciences Citation Index® (ssci), Social Scisearch®, scopus, Sociological Abstracts (csa Illumina), pio (Periodical Index Online, Reino Unido), iresie (México), icist (Canadá), hedbib (International Association of Universities - Unesco International Bibliographic Database on Higher Education), swetsnet (Holanda).

■ Journal evaluation systems: Journal Citation Reports/Social Sciences Edition (jcr), European Reference Index for the Humanities (erih), Latindex (Iberoamericana), scimago Journal & Country Rank (sjr), resh, Difusión y Calidad Editorial de las Revistas Españolas de Humanidades y Ciencias Sociales y Jurídicas (dice), carhus plus+, Matriu d’Informació per a l’Avaluació de Revistes (miar), Clasificación Integrada de Revistas Científicas circ( ).

■ Directories: Ulrich’s Periodicals Directory.

■ National catalogues: Consejo Superior de Investigaciones Científicas (csic-isoc), Red de Bibliotecas Universitarias (rebiun), Centro Nacional de Innovación e Investigación Educativa (Ministerio de Educación, Cultura y Deporte), Catálogo Colectivo de Publicaciones Periódicas en Bibliotecas Españolas (Ministerio de Educación, Cultura y Deporte).

■ International catalogues: WorldCat (usa), Online Computer Library Center (usa), Library of Congress (lc), The British Library Current Serials Received, King’s College London, Catalogue Collectif de France (CCFr), Centro de Recursos Documentales e Informáticos de la Organización de Estados Iberoamericanos (oei), copac National, Academic and Specialist Library Catalogue (United Kingdom), sudoc Catalogue du Système Universitaire de Documentation (France), zdb Zeitschriftendatenbank (Alemania).

Revista de Educación does not necessarily agree with opinions and judgements maintained by authors

6 Index

Research

Jordi Longás Mayayo, Roser de Querol Duran, Irene Cussó Parcerisas & Jordi Riera Romaní: Individual, family, school and community: multidimensional supports for school success in contexts of poverty...... 11

María del Mar Lorenzo Moledo, María José Ferraces Otero, Cruz Pérez Pérez & Concepción Naval Durán: The professor in the Service-Learning: explanatory variables...... 37

Ana María Ruíz-Ruano García, María del Pilar Casado Belmonte & Jorge López Puga: Bayesian analysis of barriers towards entrepreneurship at university...... 61

Rubén Fernández-Alonso, Pamela Woitschach & José Muñiz: Rubrics do not neutralize Raters’ effects: A many-faceted Rasch model estimation. 85

M.ª Elena Parra-González & Adrián Segura-Robles: Scientific production about gamification in education: A Scientometric analysis...... 109

Juan Francisco Ruiz-Hidalgo, Mª Elena Herrera Beltrán y Mª Victoria Ve- lasco: Calculus tasks in Spanish University entrance exams...... 133

Paula González-Vallinas, Julián Librero, Salvador Peiró & José Luis San Fabián: The impact of relative age on results in language and mathematics in primary education in Asturias, Spain...... 159

Joaquín González-Cabrera, Javier Tourón, Juan Manuel Machimbarrena, Ana León-Mejía & Mónica Gutiérrez-Ortega: Exploratory study on bullying in gifted students: prevalence and psychological wellbeing...... 181

Reviews...... 207

Research

Individual, family, school and community: Multidimensional supports for school success in contexts of poverty1

Individuo, familia, escuela y comunidad: apoyos multidimensionales para el éxito escolar en entornos de pobreza

DOI: 10.4438/1988-592X-RE-2019-386-425

Jordi Longás Mayayo Roser de Querol Duran Irene Cussó Parcerisas Jordi Riera Romaní Universidad Ramón Llull

Abstract The paper focuses on analysing the school success of secondary school students living in contexts of poverty through a multiple case study in five Spanish cities. The information has been obtained using semi-structured interviews with the students, their family members, teachers or tutors and community agents. In total, we have triangulated the information from 102 interviews with the aim of ascertaining the factors that influence students’ school success, such as their individual strengths, their trusting relationships at school and in their family, or the community support they received. The results show that 57% of the students had support in all or at least three of the dimensions studied (individual, family, school and community). This multidimensional support enables young people to develop resilience, even though factors hindering school success were found

(1) Project funded by Spanish Ministry of Economy and Competitiveness. Secretary of State for Research, Development and Innovation. Subprogram of non-oriented Fundamental Research Projects (Spain). Call 2012. Ref.: EDU2012-39497-C04-01.

Revista de Educación, 386. October-December 2019, pp. 11-35 11 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

in 70% of the cases. The findings open up new approaches to promoting equity and inclusion policies beyond the school.

Key words: child poverty, school success, community support, educational equity, multiple case studies.

Resumen El artículo presenta un estudio de casos múltiple que aborda la comprensión del éxito escolar de jóvenes en situación de pobreza y vulnerabilidad social. Se analizan en profundidad las trayectorias de éxito de 30 estudiantes que han finalizado la Enseñanza Secundaria Obligatoria en 5 ciudades españolas diferentes. Para obtener la información de cada caso se han realizado entrevistas semi-estructuradas a los propios estudiantes, a un tutor/a o profesor/a referente de su centro escolar, a un familiar y a un referente comunitario próximo. En total se ha triangulado la información de 102 entrevistas, lo cual ha permitido comprender las trayectorias de éxito escolar que se explican tanto por las fortalezas individuales como por las relaciones de confianza, los apoyos personalizados y el acompañamiento socioeducativo de los diferentes agentes implicados en la educación de los jóvenes. En el 57% de los casos se han identificado factores de apoyo al éxito escolar en todas o en 3 de las dimensiones estudiadas (individual, familiar, escolar y comunitaria). Es esta combinación de apoyos lo que permite estos jóvenes ser resilientes, a pesar de que en un 70% de los casos hemos encontrado factores inhibidores del éxito escolar. La identificación de factores de apoyo multidimensionales para el éxito escolar abre puertas a nuevos enfoques para elaborar las políticas educativas de equidad e inclusión más allá de la escuela.

Palabras clave: pobreza infantil, éxito escolar, apoyo comunitario, equidad educativa, estudio de casos múltiple.

Introduction

Since 2007, the number of families with children living in poverty has increased considerably in Spain. In this period, the context of the economic crisis has also had repercussions on worsening children’s living conditions and has revealed poverty to be a structural problem in our country. Job instability, difficulties accessing housing, transformations in

12 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty the family and the low impact of family and social policies are just some factors which explain children’s increased vulnerability due to poverty. In 2016, 28% of the Spanish population lived at risk of poverty or social exclusion, according to the AROPE indicator, a figure that reached 33% for children under the age of 18 (more than 2.7 million children and teenagers), according to figures from Eurostat (2018). Both rates are above the European Union average (EU-28). Specifically, according to Flores (2016), young people in the 12-17 age range, which dovetails with compulsory secondary education, are the most vulnerable. According to Flaquer, Almeda and Navarro (2006), equal opportunities early in the life cycle is a crucial determinant in people’s futures. In a democratic society, it is unacceptable that children suffer from exclusion processes that irreversibly condition their future. In this sense, numerous studies show the association between parents’ and children’s living conditions, confirming that equal opportunity is more an aspiration than a reality. Children who live in contexts of poverty or social exclusion will find it much harder to improve upon their parents’ socioeconomic status, thus perpetuating intergenerational transmission of poverty (ITP) (Flores, 2016) or social immobility, understood as a high proportion of children who remain in the same social class as their parents (Marqués & Herrera, 2010). The field of sociology of education has used different theoretical approaches to study the role played by education in lowering or maintaining inequalities related to family background. Since Bourdieu and Passeron (1977) in their classic reproduction theory, today we continue to witness the association between the educational and economic level of the parents and their children’s academic performance, and this is still a key factor conditioning both success and failure (Bernardi & Cebolla, 2014; MECD, 2013; Gil-Flores, 2011; Moreno, 2011; Calero, Choi & Waisgrais, 2010). The concepts of educational success and failure refer to factors that go beyond marks, although the majority of studies reduce school success/failure to this, as seen in the previous literature review which we conducted (Carrillo, Civís, Andrés, Longás, & Riera, 2018). The indicators repeatedly used to measure it are failure to attain the basic competences, repeating grades, failure to graduate from compulsory secondary school and dropping out of school. In Spain, recent data for these indicators show a very worrisome situation: 1) the PISA reports estimate that one-

Revista de Educación, 386. October-December 2019, pp. 11-35 13 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

third of 15-year-olds do not acquire basic competences in reading, maths and the sciences (UNICEF Research Centre, 2017); 2) it is estimated that 53% of the students in disadvantaged contexts repeat some grade throughout their schooling, highlighting the existence of inequalities in the life courses of students according to their socioeconomic background (OECD, 2014); 3) the overall non-graduation rate in compulsory secondary school was 22% in academic year 2014-15, that is, more than one-fifth of the youths did not graduate at the expected age (MECD, 2017); 4) the early school drop-out rate among youths aged 18 to 24 from the lowest income quintile increased from 28% to 36% in the period 2008 to 2015 (Save The Children, 2016). Dropping out of school and failing to certify the minimum competences required in our society limits their job insertion and increases the risk of social exclusion (Sarasa & Sales, 2009). Beyond the material limitations that condition educational success resulting from the situation of poverty, the affective and relational components have also been shown to be some of the factors that most determine school performance (Lozano, 2003). Family instability is often associated with the appearance of potentially traumatic life events (Pereda, 2002) and the generation of learned defenceless (Seligman, 1975), which are highly invalidating. Likewise, belonging to social networks with a noticeably vulnerable profile also increases the likelihood of suffering from stressful life events and contributes to perpetuating ITP (Muñoz, Vázquez, & Vázquez, 2003). However, even though the study of ITP can suggest an apparent determinism between poverty and school failure, research confirms that this is a complex, multidimensional phenomenon in which factors that facilitate school success also exist (Carrillo et al., 2018). Specifically, there is empirical evidence on the influence of individual factors in school failure/success (Berliner, 2009), with a particular emphasis on responsible personality structure (Paul & McCrae, 2008), as well as family factors (Hernando, Oliva, & Pertegal, 2012; Jeynes, 2007), school factors (Flecha & Puigvert, 2002; Flecha, García, Gómez, & Latorre, 2009) and community factors (Hatcher & Leblond, 2001). Specifically, positive expectations towards education emerge as an important driver of success (Pàmies, 2013) and tend to be arise as mentoring and educational support allow youths to set their academic and professional goals (Hattie, 2017; Santana, Feliciano, & Jiménez, 2016). There is recent literature analysing the systemic relationships among these factors and the positive effects of community action or coordinated

14 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty socio-educational action on the development of educational resilience and school success (Abajo & Carrasco, 2004; Moliner, 2008; Pedró, 2012; Díaz-Gibson & Civís, 2011; Civís & Longás, 2015; Díaz-Gibson, Civís, Longás, & Riera, 2017; Ruiz, Calderón, & Juárez, 2017). They all open up the possibility of designing new strategies to promote success (Smith, 2003), considering the principle of educational co-responsibility (Riera, 2007), the school-community connection from a holistic standpoint (Santalucía & Cisi de Solari, 2014; Bonal, Castejón, Zancajo, & Castel, 2015), the development of social capital (Lin, Cook, & Burt, 2001) and the national and local articulation of equity policies (Assiego & Ubrich, 2015). According to Escudero, González and Martínez (2009), students’ personal, emotional and biographical dimensions tend to remain in the shadows in studies on school failure. However, researching students’ school experience should enable us to understand the contextual conditions and how youths design their life courses and construct their identities based on their experiences (Calderón, 2014). For this reason, the objective of the study presented here is to analyse the conditioning factors of school success in secondary students living in situations of poverty and to understand how these factors work by analysing school trajectories as narrated by the students themselves.

Method

The interest in knowing the meaning and interpretation of the conditioning factors associated with school success from the standpoint of the students themselves explains the qualitative orientation of this study (Maxwell, 2004; Vasilachis, 2006). It is presented in multiple case studies (Neiman & Quaranta, 2006) because this allows for an in-depth analysis of the different school experiences based on the stories of the youths themselves and key agents in their milieu. In this way, we have sought to learn about the particularities of each life story, which would have been difficult to capture in a quantitative study (Stake, 1998), in order to obtain “local” causal explanations referring to specific processes and contexts which help to verify or develop theories around the object of study (Miles & Huberman, 1991; Maxwell, 2004).

Revista de Educación, 386. October-December 2019, pp. 11-35 15 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

Sample

The sample is comprised of 30 cases of youths living in situations of vulnerability because of relative poverty, as established by the threshold of the 2012 Living Conditions Survey (INE, 2013), and with successful school records. School success was considered having graduated from compulsory secondary school with good academic performance on average (at least 6 points out of 10 or higher) and studying or starting post-compulsory education programmes. The participating youths were between the ages of 16 and 22 when the interviews were held (26.7% were 16 years old, 26.7% were 17, 23.3% were 18, 10% were 19 and 6.7% were 21 and 22, respectively). The mean age was 17.6 for girls and 17.9 for boys. To ensure the diversity of the sample, which allows analytical generalisation albeit without purporting statistical representativeness (Coller, 2000), the inclusion criteria were to combine residency in different cities (23% Barcelona, 17% Murcia, 20% Palma de Mallorca, 20% Seville, 20% Tenerife) with the 4 sociodemographic variables which correlate with educational success according to Enguita, Mena and Riviere (2010): gender (50% males and 50% females), family background (73% Spain, 27% foreign) type of household (27% single-parent, 60% two-parent and 13% other situations) and maximum educational level attained by a parent (57% no education or primary school, 43% with secondary school or higher education).

Instruments

For each case, information was collected from the student’s academic record, and 102 semi-structured interviews were held (30 with students, 29 with parents or guardians, 27 with reference teachers in their schools and 16 with reference community agents, when applicable). The interviews were divided into 4 sets of questions: (1) characteristics of the case (student) and their school career (past, present and future expectations); (2) family’s role and expectations; (3) role of the school and teachers in the learning process; and (4) student’s relationship with their milieu, including the role and expectations of the community agent.

16 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

Procedure

The fieldwork was conducted between June and September 2014, as this was believed to be the ideal time during the academic year to collect relevant information. Each student was accessed openly through the assistance of local socio-educational agents, who helped select them based on the inclusion criteria described above. The members of the sample and the participants in the research were guaranteed the confidentiality and anonymity required for this kind of research and signed the corresponding informed consent form. The interviews were recorded and transcribed so they could later be analysed. The analysis process sought to ensure the consistency and transparency of the results bearing in mind the credibility criterion of the study (Dorio, Sabariego, & Massot, 2004; Mendizábal, 2006); triangulation strategies were used at different points in the process and the categories of analysis were designed and restructured according to the criteria of dependency and confirmability (Guba, 1981). Initially, the sources of information for each student were triangulated. After that, the information was analysed via triangulation among the researchers, such that the cases were randomly distributed among 3 teams of 2 researchers. Finally, the different cases were analysed together using the comparison by similarity strategy in order to understand the common support factors in the cases of school success studied (Yin, 2014). The participants’ narratives were analysed with the assistance of NVivo10 qualitative data software. Each group categorised the information from the cases by assigning units of text from the interviews to the categories established in the design of the study and identifying emerging categories. Thus, a flexible or mixed design (deductive and inductive) was used, starting with an analysis structure devised based on the literature review, which was open to the interactive generation of knowledge based on the data obtained (Glaser & Strauss, 1967; Blumer, 1982; Mendizábal, 2006). The restructuring of the categories of analysis based on shared reflection on their meanings (Barber & Walczak, 2009) enabled a matrix of fields and categories of analysis to be established (Table I) to compare the concordant or discordant discourses among the different agents and cases, as well as the common meanings which may be generalisable.

Revista de Educación, 386. October-December 2019, pp. 11-35 17 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

TABLE I. Categories by dimensions of analysis

Num- Num- Total ber of ber of Dimen- number % of the Categories male female sions of refer- total refer- refer- ences ences ences Individual 340 562 902 33 Skills and competences 34 43 77 3 Personality: neuroticism, extraversion, openness, kindness, responsibility and 103 221 324 12 resilience Personal study: learning style, difficul- 77 103 180 7 ties, space and time for schoolwork Life habits 64 96 160 6 Future plans 49 72 121 4 Assessment of education 13 27 40 1 Family 278 522 800 30 Oversight of schooling 68 97 165 6 Models in the family and child-rearing 14 39 53 2 style Family’s expectations of the youth 31 39 70 3 Family’s assessment of education 21 36 57 2 Family-school relationship 33 45 78 3 Family’s relationship health 30 73 103 4 Family status: structure, sociocultural context, work and economic status, 81 193 274 10 housing and health. School 249 409 658 24 School record 82 99 181 7 Youth’s integration at school: relationships with peers, participation in and 44 83 127 5 assessment of the school Teacher-student relationship 37 72 109 4 Teachers’ expectations of the youth 22 23 45 2 School: sociocultural context, educa- 64 132 196 7 tional curriculum and organisation

18 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

Commu- 187 165 352 13 nity Relationship with community referent 29 16 45 2 Educational oversight of the commu- 62 20 82 3 nity referent Expectations of the community refer- 12 8 20 1 ent Peer relationships outside of school 42 46 88 3 Local participation: free-time, educa- 42 75 117 4 tional or social activities Total 1.054 1.658 2.712 100

Source: Authors

Of all the interviews, 2,712 units of text were coded in the 4 fields (902 individual, 800 family, 658 school and 352 community). The overall analysis of these units enabled us to identify some social handicap conditions and recurring factors that hinder or foster school success. In view of the emergence of potentially traumatic life events, we chose to use the classification by Pereda (2002), while adding a few more categories to record other situations which may be stressful. Once the cases were analysed, the strengths of the school success factors were evaluated for each dimension according to the importance that the interviewees attributed to them. By doing so, we made an approximation of the concept of multidimensional support and its systemic value.

Results

The results of the study are presented in two sections: 1) analysis of life contexts of the participants and factors hindering school success; and 2) analysis of factors supporting school success.

Drudgery of poverty and factors hindering school success

In all the cases, situations of poverty were identified that lowered students’ access to material resources and opportunities or even deprived

Revista de Educación, 386. October-December 2019, pp. 11-35 19 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

the students of them entirely, with a potential to affect their education. Economic status affects being able to afford school costs (textbooks and school supplies, extra activities), engagement in extracurricular activities (English, sports, tutoring, etc.) and the availability of space and resources to study at home (computer, Internet access, etc.). This reality tends to come with a complex system of feelings and effects which directly condition school performance and more indirectly affect the motivation to study. What stands out in the narratives is the perception of a kind of stigmatisation associated with poverty, such that in addition to the material barriers, the youths also feel symbolic barriers. In the majority of cases, they are aware that they are accessing the educational system without having the same opportunities as their classmates. In a subtle way, material deprivation implies creating social relations based on feelings of “inferiority”, a kind of oppression that is particularly felt in adolescence, a stage of self-affirmation and the construction of one’s identity.

My mother’s economic situation made me feel inferior and it affected me a lot (…) if some people realised it, they made fun of me. Not everyone did, but it made me feel insecure. (Youth, C17, Tenerife)

In 37% of the cases, the families and school and community agents expressed limitations in their expectations of the youths’ future due to their poverty. They clearly stated that economic status can be an insurmountable obstacle for youths to continue studying, regardless of their good performance. The effects of poverty are also felt in the family’s day-to-day lives. The stress that parents feel when trying to make ends meet every month, some of them even facing eviction, often leads to a gradual deterioration in the family dynamic. Likewise, in the cases studied, we found a high unemployment rate in the families (33.3% with no income from work, 47.7% one-income families; 20% two-income families). Many parents experience long-term unemployment and unstable jobs, which forced them to hold down several jobs or get involved in the underground economy. All of this means that a feeling of insecurity prevails in the families’ perception, regardless of whether or not they work. Some youths are clearly aware of the harsh experience of instability generated by their parents’ lack of work or unstable jobs and try to help out the family economically by doing paid jobs, taking on responsibilities and family roles which are beyond their age.

20 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

The parents’ education also arises in the participants’ discourses, as they acknowledge the sociocultural handicap that their educational level entails for their children. This particularly affects them because of the impossibility of them understanding and/or helping their children in their learning processes, the lack of direct stimuli in the family or the lack of referents of school success within the nuclear and extended family. The lack of referents within the neighbourhood was also mentioned as a difficulty in some cases. This absence of role models also extends to peer relationships. Some youths claim that they found no stimuli to study in their social networks due to their friends’ lower educational level and/ or motivation. Others directly recognised the negative influence their peers had on them.

He wants to show the people in the neighbourhood that you can do something else, like study, because this means that in the future he’ll get a good job; he wants to be an example for the neighbourhood. (Mother, C20, Murcia).

The friends I have in the neighbourhood, probably the one with the most education has a secondary school certificate, that’s it. So of course, I’m kind of like the weird one. (Youth, C9, Mallorca)

What is noteworthy is the large number of experiences identified as highly stressful situations in themselves which affect emotional wellbeing and can be described as potentially traumatic. A total of up to 72 experiences of this were tallied (Table II) which have affected up to 70% of the cases (N=21), coupled with the difficulties stemming from economic poverty, a depressed sociocultural environment and other school-related or typically adolescent problems (fights with siblings, emotional instability, lack of direction, etc.). It is frequent for the same student to experience several of these situations. The distribution or concentration of experiences of this type is as follows: 13.3% of the cases have 9, 7 or 6 life stressors; 23.3% of the cases have 4 or 3 stressful life situations; and 33.3% of the cases have 2 or 1 experiences of this kind. Table II shows the different stressful life experiences or events which were identified. Negative experiences related to health have affected 46.7% of the youths in the sample, who have experienced serious family illnesses, experiences of death of close family members and/or illnesses themselves. Forty percent of the cases have experienced undesired and

Revista de Educación, 386. October-December 2019, pp. 11-35 21 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

potentially destabilising changes, such as a change in country in the case of immigrants, moving home and a new school, which bring pain, uncertainty and efforts to adapt. A total of 20 experiences of instability within the family were identified, caused by ruptures, regrouping or conflicts in family life, which affect 11 cases of 36.6% of the sample. Twenty percent of the students had experienced abuse or direct aggression, either physical or psychological, in its different manifestations, such as harassment or bullying at school, gender violence at home, sexual abuse by their father and abortion because of unwanted pregnancy. Another situation we have classified as stressful is sudden poverty, which has occurred in 23.3% of the families in the sample. Other families also live in situations of poverty, sometimes even more severe, such that the stress entailed by having limited access to many resources and living with uncertainty about the future affects all the students. However, the particularity of these cases is that they started in the middle or lower- middle class and were forced to change their lifestyle and learn how to live with instability because of the effects of the economic crisis.

TABLE II. Distribution of stressful life experiences

Type of experiences No. Negative health experiences 19 Family illness 8 Death of loved one 6 Illness (self) 5 Undesired and potentially destabilising 17 changes Immigration 5 Moving home 7 New school 5 Family instability 20 Abandonment 6 Adoption or fostering 3 Living away from the family 5 Divorce or separation 5 Parent in prison 1

22 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

Experiences of abuse and direct aggression, either physical or psycho- 9 logical Harassment or bullying from peers 5 Gender violence at home 2 Sexual abuse by father 1 Abortion due to unwanted pregnancy 1 Sudden poverty 7 Total 72

Source: Authors

Multidimensional supports to achieve school success in contexts of poverty

We have identified the supports the youths have received in each area of analysis. The narratives acknowledge them as factors explaining their success, and sometimes as factors that offset other difficulties. These supports are distinguished from what we call conditions of possibility which could favour the youths’ school success because they are attributed direct action, which is somehow a crucial cause explaining it.

■ Individual

The individual strengths of the youths oriented explicitly at success at school and in education are often presented as both a result and a factor of support for their successful careers. On the one hand, they show attainment of a life trajectory directly influenced by the other factors that facilitate success which we have identified in the family, school and community environments. Yet they also simultaneously emerge as protective factors that explain the success itself. The participants’ discourses particularly stress the youths’ own cognitive capacities, personal features and future plans, which are very powerful factors that help them overcome adversity:

From a very young age, I saw that education is the only way to get ahead; it was like my only lifesaver. (Youth, C5, Barcelona)

Revista de Educación, 386. October-December 2019, pp. 11-35 23 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

In terms of personality, what stands out is the capacity for personal and educational resilience of the youths interviewed. In 57% of the cases, the youths’ capacity to deal with difficult situations and “fight” to achieve their goals is mentioned explicitly. Another noteworthy personality characteristic is responsibility, as defined in the Big Five model (Paul & McCrae, 2008) as self-awareness and self-esteem, organisation, a sense of duty, a need for achievement or determination, self-discipline and conscientiousness. Responsibility in one of these facets appears in all the cases in the sample. Finally, a crucial lever in success according to the youths’ narratives are the expectations of education to help them achieve their life plans. Fifty-three percent of the youths in the sample value education as an investment in the future, which sparks their interest in learning and shows that when the social milieu does not push them to have horizons beyond the neighbourhood itself, looking outward and connecting with other milieus can counter this. In some cases, these clearly defined future expectations and life plans emerge in response to the very situation of deprivation and vulnerability. Specifically, in 5 cases (17% of the sample), the main factor explaining school success is that the youths have taken refuge in their studies to compensate for problematic family situations. Having a better future is what makes their life courses meaningful.

■ Family

For the youths, regardless of their parents’ level of education, having their family monitor their school activity is particularly important because it fosters school inclusion. This direct support for education is perceived not so much in help with doing schoolwork as in monitoring their education (40% of the cases), keeping track of their calendar and homework and showing an interest in what is happening at school. Emotional support from the family is also valued when the youths feel “overwhelmed” by difficulties in their schoolwork or frustrated by not achieving the academic results they wanted. More directly, positive parenting practices in the family are important support for school success in 60% of the cases. Good communication between family members, time shared together, trust and/or mutual support in some cases are crucial to offsetting certain difficulties related to the family economic situation,

24 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty stressful life events or risks detected in other areas of analysis. In some cases, what also stands out is the strength of the family environment, especially the role of some mothers, who convey positive values which serve as an invaluable lever of support in the construction of the youths’ resilient personalities:

I always tell her: a warrior has to dry her tears and keep going. (Mother, C18, Tenerife)

It was found that the trust deposited in the youth is more obvious when the family agent interviewed expressed a positive view of education (53% of the sample) and projects positive future expectations, either by praising their children’s ability to achieve their goals or by valuing the possibilities of social mobility (“becoming someone” or “going far”).

■ School

At school, the most valued and effective lever is teachers’ personal support and high expectations (80% of the cases). The narratives show that the existence of one teacher that keeps watch over the youths both educationally and personally leads them to feel “pushed” to improve:

If it hadn’t been for her (teacher) I think I would have foundered. (Youth, C8, Mallorca)

In many cases, this personalisation of educational mentoring is not always associated with instruction, which is regulated by the school, but instead appears with certain educators with whom they have forged strong bonds. They primarily act as role models and/or project positive future expectations for the youths. In 2 cases of dire vulnerability (lack of support outside school), the personal commitment of some teacher was the determining factor. The other major lever of school success at school is positive peer relations. These can take on different guises: academic support among peers, integration into the class, friendships with some classmates that extend outside school and the positive influence of pro- school discourses in the peer group.

Revista de Educación, 386. October-December 2019, pp. 11-35 25 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

■ Community

In this dimension, which categorises activities and supports found outside the school, extremely influential supports of school success were identified by leaders of entities involved in socio-educational activities and professionals in different extracurricular activities in 70% of the cases. They can be a taekwondo instructor, a music teacher or a village priest:

He’s more an educator than just a taekwondo teacher (Youth, C2 Barcelona).

Their stability or permanence during the youths’ school years, their ability to offset some family limitations and the human quality of the bonds are the most striking explanations of how this community support works. Beyond the positive impact of the relationships established with the referents, mere participation in the community is also a lever that facilitates success. One the one hand, local socio-educational entities offer guidance and academic support, and personal mentoring for the youths and their families depending on their needs (school, psychological and economic support, job orientation, leisure and free-time activities, etc.). Meanwhile, engaging in educational activities (sports, theatre, music, organised leisure, etc.) is an opportunity to forge new social relations which are stimulated by sharing something that is very personally valuable or by the self-esteem and confidence that the youths practising these activities develop. Considering the quality, consistency, persistence or importance that the participants assign to the support factors for their success, the presence of factors that strongly influence success in the 4 dimensions of analysis was assessed. Table III shows a summary of their distribution according to the number of dimensions that incorporate strengths and their combination. It is worthwhile to assess the distribution of the support factors by dimensions in order to determine their systematic nature and the hypothetical interrelations among them: in 57% of the cases, the multidimensional nature of the levers of success is recognised (which affect 3 or 4 dimensions); in 33% of the cases, a two-dimensional distribution is recognised; and in 10% of the cases, factors that support

26 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty school success are only identified in one dimension. We can see that success is attributed to both individual strengths (67% of the cases) and to the support received from the family (63%), school (57%) and community (67%). It is worth noting that a high number of youths have personal strengths that help them achieve school success, but in all of them we can also find multidimensional or two-dimensional support from agents within each student’s milieu, be it from the family, school or community. In contrast, in 9 of the 10 cases where individual strengths do not exist, there is a greater presence of support from the community, which in some cases combines with major support in school and the family.

TABLE III. Distribution of supports by dimensions

Supports according to area of Total dimensions of support Case code analysis per case Indiv. Family School Comm. 4 3 2 1 2      18      16     1     4     7     19     3     14     28     21     27     29     9     13     23     25    

Revista de Educación, 386. October-December 2019, pp. 11-35 27 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

6    17    5    12    8    20    22    26    30    24    15   10   11   Total cases 20 19 17 20 2 15 10 3 % of cases 67 63 57 67 7 50 33 10

Source: Authors

Discussion and conclusions

The research enables us to gain an in-depth understanding that the poverty which affects children and adolescents is not only material poverty but also has social handicapping effects such as insecurity, low access to positive models, etc. We can conclude that the youths in the sample live in conditions that are largely unfavourable due to two kinds of effects: a) adverse, directly aggressive or even potentially traumatic conditions (Pereda, 2002); and b) conditions of social impoverishment which lower or eliminate their access to opportunities for greater educational development which in themselves are major social inequalities. Even though all cases share the invisible barriers of economic poverty, the importance of relational wellbeing and the construction of positive relationships with adults and/or peers from the family, the school and community organisations are also associated with their school success (Lozano, 2003). Beyond individual strengths or weaknesses, the most determining factors of success are the people who mentor or act as role

28 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty models. Sometimes the relationship itself, the quality of the bond and the commitment with others is what leads the youth to persist and seek success. In some cases, respect and admiration for someone else is what induces them to imitate this person. And in almost all cases, the person- mentor who guides, mentors, accompanies and sustains the youth is acknowledged. The main educational role is school mentoring meant as offering security, generating trust, developing self-esteem and helping students construct future expectations. As Flores (2016) notes, the different personalised supports which the youths have received do not act exclusively as factors preventing academic careers doomed to failure and exclusion; instead, they also act as factors of integration because they offset the shortcomings with which the youths begin. The existence of positive relationships not only is a personal support and solidarity structure but also acts as a catalyst of their own resources and capacities. Our study found evidence of the strength of the school-family- community alliance. In the cases where the family or school supports are not valued as levers facilitating school success, there is always a community referent with whom the youth has forged stable, high- quality bonds, showcasing the role of social education and third-sector organisation from the perspective of school success. It can be claimed that the resilient processes developed by the youths in the sample can be explained by the multidimensional interaction of individual strengths and support factors in the family, school and community. Feeling one’s life course and building one’s future in a shared way emerges as a driving force in the struggle to overcome educational inequalities. As Ruiz et al. (2017) state, resilience goes from being a quality to being a community educational action in which the possibility of social transformation is constructed. This capacity to overcome the social adversity associated with poverty, which is more possible when the community weaves a systemic web of complex interactions that explicitly and implicitly support children’s holistic development, matches the theories of social capital and its impact on educational and school success (Lin et al., 2001). The youths’ connection to a variety of people and organisations opens up new educational opportunities for them, as the barriers limiting this access are an important factor in educational inequality. In this sense, policies geared at educational improvement should incorporate the core objective of building local community social

Revista de Educación, 386. October-December 2019, pp. 11-35 29 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

capital by ensuring access to libraries, IT resources, educational support services, school-setting projects, cultural and leisure activities, sports activities, etc. All of these community resources together should rise to the challenges of education by incorporating the vantage point of all the socioeducational agents in order to go beyond their individual action, hierarchies, sectorisation and the mechanisms of the educational system for the sake of greater social efficiency (Santalucía et al, 2014; Díaz- Gibson et al., 2011; Assiego et al, 2015; Bonal et al., 2015). This study found that in the cases of lower individual strengths, the presence of community support factors is crucial. What clearly emerges is the importance of having social networks or socioeducational resources that promote school success near the youths, somehow local or regional ones. In this sense, community social capital can be understood as a kind of social and educational guarantee or support which expands the provision of education that the (universal) educational system represents. For all of these reasons, it would be wise to consider the value of articulating community development projects that solidify socioeducational networks and intersectoral cooperation because of their benefits in improving academic performance in the middle and long term (Díaz-Gibson et al., 2017) and their effects on lowering ITP (Flores, 2016). In the cases studies, it was found that successful academic careers are not linear but instead tend to follow processes of crisis and restructuring. The shadows of fragility and uncertainty are still cast over the youths’ evolution. Hope in the future is contained by traumatic experiences or is assumed to be threatened by a lack of resources (economic and time), which can prevent them from accessing higher levels of education. This is one of the limitations of our study. We have chosen youths who at a given point in time had successful careers which we have defined in terms of graduation and continuing with post-compulsory studies. However, this attainment is only the first phase in educational success. Hence, we would suggest the possibility of conducting a subsequent longitudinal study by once again interviewing those youths who want to continue participating in the study.

30 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

Bibliographic References

Abajo, J.E & Carrasco, S. (ed.) (2004). Experiencias y trayectorias de éxito de gitanos y gitanas de España. Madrid: CIDE/ Instituto de la Mujer. Assiego, V. & Ubrich, T. (2015). Iluminando el futuro. Invertir en Educación es luchar contra la pobreza infantil. Madrid: Save the Children España. Barber, J. P. & Walczak, K.K. (2009). Conscience and Critic: Peer Debriefing Strategies in Grounded Theory Research, Annual Meeting of the American Educational Research Association. San Diego, California. Berliner, D.C. (2009). Poverty and Potential: Out-of-School Factors and School Success. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Bernardi, F. & Cebolla, H. (2014). Clase social de origen y rendimiento escolar como predictores de las trayectorias educativas, Revista Española de Investigaciones Sociológicas, nº146, 3-22. doi:10.5477/ cis/reis.146.3 Blumer, H. (1982). El interaccionismo simbólico: perspectiva y método. Barcelona: Hora. Bonal, X. (dir.), Castejón, A., Zancajo, A. & Castel, J.L. (2015). Equitat i resultats educatius a Catalunya. Una mirada a partir de PISA 2012. Barcelona: Fundació Jaume Bofill. Bourdieu, P. & Passeron, J.C. (1977). La Reproducción: elementos para una teoría del sistema de enseñanza. Barcelona: Laia. Calderón, I. (2014). Sin suerte pero guerrero hasta la muerte: pobreza y fracaso escolar en una historia de vida. Revista de Educación, nº 363, 194-209. doi: 10.4438/1988-592X-RE-2012-363-177 Calero, J., Choi, A. & Waisgrais, S. (2010). Determinantes del riesgo de fracaso escolar en España: una aproximación a través de un análisis logístico multinivel aplicado a PISA-2006, Revista de Educación, número extraordinario, 225-256. Carrillo, E., Civís, M., Andrés, T., Longás, E. & Riera J. (2018). Condicionantes del éxito y fracaso escolar en contextos de bajo nivel socioeconómico. Revista de Estudios y Experiencias en Educación. Número especial nº2, 75-94. doi: 10.21703/rexe.Especial2_201875944 Centro de Investigaciones de UNICEF. (2017). Construir el futuro: los niños y los Objetivos del Desarrollo Sostenible en los países ricos. Report Card nº14 de Innocenti. Florencia: Centro de Investigaciones de UNICEF.

Revista de Educación, 386. October-December 2019, pp. 11-35 31 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

Civís, M. & Longás, J. (2015). La colaboración interinstitucional como respuesta al desafío de la inclusión socioeducativa. Análisis de 4 experiencias de trabajo en red a nivel local en Cataluña. Educación XXI, 18 (1), 213-236. doi:10.5944/educXX1.18.1.12318 Coller, X. (2000). Estudio de casos. Madrid: Centro de Investigaciones Sociológicas (CIS), Cuadernos metodológicos, nº 30. Díaz-Gibson, J. & Civís, M. (2011). Redes Socioeducativas promotoras de capital social en la comunidad: un marco teórico de referencia, Cultura y Educación, Vol. 23, nº 3, 415-429. Díaz-Gibson, J., Civís, M., Longás, J. & Riera, J. (2017). Projectes d’innovació educativa comunitària: ingredients d’èxit i reptes. Informes breus 64. Fundació Jaume Bofill. Dorio, I., Sabariego, M., & Massot, I. (2004). Capítulo 9: Características generales de la metodología cualitativa. In R. Bisquerra (coord.), Metodología de la Investigación Educativa, (275-292). Madrid: Editorial La Muralla. Enguita, M.F., Mena, L. & Riviere, J. (2010). Fracaso y abandono escolar en España. Barcelona: Obra Social “la Caixa”, Colección Estudios Sociales, nº 29. Escudero, J.M., González, M. T. & Martínez, B. (2009). El fracaso escolar como exclusión educativa: comprensión, políticas y prácticas. Revista Iberoamericana de Educación, nº50, 41-64. Eurostat (2018). People at risk of poverty or social exclusion by age and sex. EU-SILC 2007-2016. [ilc_peps01, 18/01/2018]. Retrieved from http://ec.europa.eu/eurostat/data/database Flaquer, L., Almeda, E. & Navarro, L. (2006). Monoparentalidad e infancia, Barcelona: Colección Estudios Sociales, nº 20. Obra Social Fundación La Caixa. Flecha, R., García, R., Gómez, A. & Latorre, A. (2009). Participación en escuelas de éxito: una investigación comunicativa del proyecto Includ- ed. Cultura y Educación 21, 183-196. Flecha, R., & L. Puigvert, L. (2002). La Comunidad de Aprendizaje: Una Apuesta por la igualdad educativa. REXE: Revista de Estudios Y Experiencias En Educación, 1, 11–20. Flores, R. (coord.) (2016). La transmisión intergeneracional de la pobreza: factores, procesos y propuestas para la intervención. Madrid: Fundación FOESSA.

32 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

Gil-Flores, J. (2011). Estatus socioeconómico de las familias y resultados educativos logrados por el alumnado, Cultura y Educación, Vol. 23, nº1, 141-154. Doi: 10.1174/113564011794728597 Glaser, B.G. & Strauss, A.L. (1967). The Discovery of Grounded Theory. Strategies for Qualitative Research. Londres: Weindenfeld and Nicolson. Guba, E.G. (1981). Criteria for assessing the trustworthiness of naturalístic inquiries. Educational Technology Research and Development (ECTJ), 29(2), 75-91. doi:10.1007/BF02766777 Hatcher, R. & Leblond, I. (2001). Education Action Zones and Zones d’Education Prioritaires. Keele University. The United Kigndom, 2001. 29. Hattie, J. (2017). Aprendizaje visible para maestros, Madrid: Parninfo (Didáctica y Desarrollo). Hernando, A., Oliva, A. & Pertegal, M.A.. (2012). Variables familiares y rendimiento académico en la adolescencia. Estudios de Psicología, 33, 51-56. INE. Instituto Nacional de Estadística. (2013). Encuesta de Condiciones de Vida Año 2012. Datos definitivos 2012. Nota de prensa 20/11/2013. Madrid: INE. Jeynes, W. H. (2007). The relation between parental involvement and urban secondary school student academic achievement: A meta-analysis. Urban Education 42, 82–110. http://dx.doi. org/10.1177/0042085906293818 Lin, N., Cook, K. & Burt, R.S. (eds.) (2001). Social Capital. Theory and Research. New Brunswick (New Jersey): Transaction Publishers. Lozano, A. (2003). Factores personales, familiares y académicos que afectan al fracaso escolar en la educación secundaria, Revista Electrónica de Investigación Psicoeducativa y Psicopedagógica, nº 1, 43-66. Marqués, I., & Herrera, M. (2010). ¿Somos más móviles? Nuevas evidencias sobre la movilidad intergeneracional de clase en España en la segunda mitad del siglo XX. Revista Española de Investigaciones Sociológicas, nº131, 43-73. Maxwell, J. (2004). Using qualitative methods for causal explanations. Field Methods, 16 (3), 243-264. MECD Ministerio de Educación, Cultura y Deporte. (2013). PISA 2012: Programa para la Evaluación Internacional de los Alumnos. Informe

Revista de Educación, 386. October-December 2019, pp. 11-35 33 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

Español. Volumen 1: Resultados y Contexto. OCDE. Madrid: Ministerio de Educación, Cultura y Deporte. MECD Ministerio de Educación, Cultura y Deporte (2017). Sistema estatal de indicadores de la educación 2017. Madrid: Instituto Nacional de Evaluación Educativa. Subdirección General de Estadística y Estudios. Mendizábal, N. (2006). Los componentes del diseño flexible en la investigación cualitativa, en I. Vasilachis (coord.), Estrategias de investigación cualitativa. Barcelona: Gedisa. Miles, M.B. & Huberman, A.M. (1991). Analise des donnés qualittaives. Recueill de nouvelles méthodes. Paris: De Boeck Université. Moliner, O. (2008). Condiciones, procesos y circunstancias que permiten avanzar hacia la inclusión educativa: retomando las aportaciones de la experiencia canadiense. Revista Electrónica Iberoamericana Sobre Calidad, Eficacia y Cambio en Educación, 6,27–44. Moreno, A. (2011). La reproducción intergeneracional de las desigualdades educativas: límites y oportunidades de la democracia. Revista de Educación. Número extraordinario, 183-206. Muñoz, M., Vázquez, C., & Vázquez, J.J. (2003). Los límites de la exclusión, Caja Madrid, Madrid. Neiman, G. & Quaranta, G. (2006). Los estudios de caso en la investigación sociológica, en I. Vasilachis (coord.), Estrategias de investigación cualitativa. Barcelona: Gedisa. OCDE. (2014) ¿Tienen más probabilidades de repetir curso los estudiantes más desfavorecidos?, PISA in Focus, 43, Septiembre 2014. Retrieved from http://www.mecd.gob.es/dctm/inee/pisa-in-focus/pisa-in-focus- n43-esp--v3.pdf?documentId=0901e72b81b30574 Pàmies, J. (2013). The impact of groupings in school. Spaces of learning and sociability for Moroccan youth in Barcelona. Revista de Educación, (362), 133–158. Paul, T. & McCrae, R. (2008). Inventario de personalidad NEO reducido de cinco factores (NEO-FFI). TEA Ediciones: Madrid (3ª Edición revisada y ampliada). Pedró, F. (2012). Políticas públicas sobre apoyo y refuerzo educativo: evidencias internacionales. Revista de Educación, número extraordinario 2012, pp. 22-45. Pereda, N. (2002). Traumatic Life Events Questionnaire (TLEQ). Copyright E. S. Kubany (1995, 1998). Reproduced by permission. Experimental Spanish Edition, Department of Personality, Assessment and Psychological Treatments. Faculty of Psychology, Barcelona.

34 Revista de Educación, 386. October-December 2019, pp. 11-35 Received: 03-05-2018 Accepted: 17-05-2019 Longás Mayayo, J., de Querol Duran, R., Cussó Parcerisas, I., Riera Romaní, J. Individual, family, school and community: Multidimensional supports for school success in contexts of poverty

Riera, J. (2007). Reflexió entorn de les coresponsabilitats socioeducatives. Per a un nou marc de relacions entre la familia, l’escola i la societat: En J. Riera y E. Roca. (coords.), Reflexions sobre l’educació en una societat corresponsable. Valls-Barcelona: Cossetània Edicions y Edu21, 13-27. Ruiz, C., Calderón, I. & Juárez, J. (2017). La resiliencia como forma de resistir a la exclusión social: un análisis comparado de casos. Pedagogía Social. Revista Interuniversitaria, nº 29, 129-141. doi: 10. SE7179/ PSRI_2017.29.09 Santalucía, P. & Cisi de Solari, O. (2014), Herramientas para la acción tutorial, Buenos Aires, Dunken. Santana, L.E.; Feliciano, L. & Jiménez, A. B. (2016). Apoyo familiar percibido y proyecto de vida del alumnado inmigrante de Educación Secundaria. Revista de Educación, nº 372, 35-62. doi: 10.4438/1988- 592X-RE-2015-372-314 Sarasa, S. & Sales, A. (2009). Itineraris i Factors d’Exclusió Social. Barcelona: Ajuntament de Barcelona. Síndica de Greuges de Barcelona. Save the Children. (2016). Necesita mejorar. Por un sistema educativo que no deje nadie atrás. Retrieved from https://www.savethechildren. es/publicaciones/necesita-mejorar Seligman, M. (1975). Helplessness: On Depression, Development, and Death. San Francisco: W.H. Freeman. Smith, R. (2003). Research and Revelation: What Really Works. In P. Smeyers and M. Depaepe (Eds.) (2003). Beyond Empiricism: On Criteria for Educational Research. Leuven: Leuven University Press, pp. 129-140. Stake, R.E. (1998). Investigación con estudio de casos. Madrid: Morata Vasilachis, I. (2006). La investigación cualitativa, en I. Vasilachis (coord.), Estrategias de investigación cualitativa. Barcelona: Gedisa. Yin, R.K. (2014). Case Study Research: Design and Methods. Thousand Oaks CA: Sage Publications.

Contact address: Jordi Longás Mayayo. Facultad de Psicología, Ciencias de la Educación y del Deporte Blanquerna. Universidad Ramon Llull. C/ Císter, 34, 08022, Barcelona. E-mail: [email protected]

Revista de Educación, 386. October-December 2019, pp. 11-35 35 Received: 03-05-2018 Accepted: 17-05-2019

University teaching staff facing Service-Learning: explanatory variables1

El profesorado universitario ante el aprendizaje-servicio: variables explicativas

DOI: 10.4438/1988-592X-RE-2019-386-426

María del Mar Lorenzo Moledo María José Ferraces Otero Universidade de Santiago de Compostela Cruz Pérez Pérez Universitat de València Concepción Naval Durán Universidad de Navarra

Abstract Introduction: The Service-Learning methodology is one of the most appropriate to adapt to changes, and new social and educational challenges in the Higher Education environment. This is an approach that promotes an experiential type of learning in which the teaching staff does not play a prominent role, but those who offer and receive the service. There are several reasons for proposing this methodology in the university classrooms, and they have to do with the improvement of the educational processes, students’ ethical and civic education, as well as with the improvement of the existing linkages between university and society. Specifically, our objective in this paper is to analyze which are the variables explaining the introduction of the Service-Learning methodology by a university

(1) Project supported nº EDU2013-41687-R. Competitive research project supported by the Spanish Ministry of Economy and Competitiveness, within the framework of the R&D&I State Program ‘Societal Challenges’: “Service-Learning and Innovation at University Level. A program to improve students’ academic performance and social capital” co-funded by the European Regional Development Fund.

Revista de Educación, 386. October-December 2019, pp. 37-60 37 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

professor. Methodology: To this end, a study involving 1903 professors from 6 Spanish universities was carried out. They were administered the “Questionnaire on Teaching Practice and Attitude of University Professors towards Innovation” (CUPAIN). Results: With the data obtained, a logistic regression was performed in order to estimate the probability that a university teacher uses Service- Learning according to sociodemographic variables (gender, age, type of subject, students enrolled, administrative situation, experience, cycle), teaching practice, engagement with the university’s social commitment, and interest in innovative teaching practices; subsequently, a multivariate analysis of variance (MANOVA) 5 X 2, and a Chi-square test were applied. Discussion and Conclusions: The data analysis allowed us to conclude that, of all the analyzed variables, the one that better predicts the use of SL by the teaching staff is their engagement with the university’s social commitment, in such a way that the probability of using this methodology is 3.52 times higher than not doing it.

Keywords: Service-Learning, university, social commitment, teaching staff, teaching practice.

Resumen Introducción: La metodología del aprendizaje-servicio es una de las más adecuadas para adaptarse a los cambios y a los nuevos retos sociales y educativos en el espacio universitario. Se trata de un enfoque que promueve un aprendizaje de tipo experiencial en el que el protagonismo no está en el profesorado sino en quienes ofrecen y en quienes reciben el servicio. Las razones para proponer esta metodología en las aulas universitarias son muy variadas, y tienen que ver con la mejora de los procesos educativos, la formación ética y ciudadana del alumnado, así como con la mejora de los vínculos existentes entre universidad y sociedad. Concretamente, nuestro objetivo en este trabajo es analizar cuáles son las variables que explican que un profesor universitario introduzca la metodología de aprendizaje-servicio en su quehacer docente. Metodología: Con este propósito, hemos llevado a cabo un estudio con 1903 docentes de 6 universidades españolas a los que se les ha aplicado el “Cuestionario sobre Práctica docente y Actitud del profesorado universitario hacia la Innovación” (CUPAIN). Resultados: Con los datos obtenidos realizamos una regresión logística con la finalidad de estimar la probabilidad de que un profesor universitario utilice o no el aprendizaje- servicio según variables sociodemográficas (sexo, edad, tipo de materia, alumnos matriculados, situación administrativa, experiencia, ciclo), la práctica docente, el acuerdo con el compromiso social de la universidad y el interés por prácticas docentes innovadoras; y, posteriormente, un análisis de varianza multivariado (MANOVA) 5 X 2, y una prueba chi-cuadrado. Discusión y conclusiones: El análisis de los datos nos permite concluir que de todas las variables analizadas la que mejor predice el empleo del ApS por parte de los docentes es su acuerdo con el compromiso social de la universidad, de tal forma que la probabilidad de utilizar esta metodología es 3.52 veces mayor que no hacerlo.

38 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

Palabras clave: aprendizaje-servicio, universidad, compromiso social, profesorado, práctica docente.

Introducción

21st-century society is experiencing very rapid changes in significant aspects such as increasing people’s mobility, the need to learn throughout life, the growth in cultural diversity, the advances in information and communication technology, the exponential growth of knowledge or the new scenarios arising for social relationships, to name but a few. Universities cannot remain on the outside of this new social reality and have to face these successive challenges without delay. As pointed out by Santos Rego, Sotelino, and Lorenzo (2015), the role of universities cannot be depleted in the conventional tasks of a higher education institution, focused on the coordination of teaching programs and research lines. They have to reflect on their civic mission and be at the forefront of programs and projects, being able to change and improve their surrounding environment and contributing to a more inclusive society. Society has been calling for a model of university that is willing to go beyond the training of professionals; it requires a university committed to the community of origin, which is asked to train a critical, responsible citizenship that works to overcome existing injustices and inequalities, seeking the common good (Martínez-Martín, 2016). To this end, university should adopt new methodological proposals, many of which are developed in different educational environments, which will imply an improvement of the training options available to students (Piqué & Forés, 2013). It is only in recent years that this request has become more firm. Thus, along with the traditional teaching and research functions, particular emphasis is being placed on the so-called “third mission of the university”. The aim is that part of what is received from society returns to it, in accordance with the citizens’ requirements (Yáñez-Aldecoa, Okada, & Palau, 2015). This implies a constant dialog and communication between the university community and civil society; the focus of interest is to identify needs and seek ways of collaboration.

Revista de Educación, 386. October-December 2019, pp. 37-60 39 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

In this sense, the process of convergence and adaptation to the European Higher Education Area (EHEA) was a good opportunity for reflection, renewal and improvement (Santos Rego, Jover, Naval, Álvarez, Vázquez, & Sotelino, 2017; Naval and Arbués, 2018). Article 64.3 of the Royal Decree 1791/2010 of December 30, which approves the Statute of the University Student, specifies that universities should favor social and citizen responsibility practices by combining academic learning in different degree programs and the provision of services to the community, focused on the improvement of the quality of life and social inclusion. On the same vein, the Spanish Qualifications Framework for Higher Education (Royal Decree 96/2014, of February 14) foresees as a learning outcome the students’ ability to make an ethical reflection on their field of study, which implies the need to define strategies and learning methodologies, ensuring students’ ethical education. On the one hand, the University Strategy 2015 on the modernization process of Spanish universities encouraged including in their training model practical activities integrating the preparation of students for a career, along with the corresponding exercise of social responsibility (Ministry of Education, 2011). In 2015, CRUE approved a document for the institutionalization of Service-Learning as a teaching strategy within the framework of the academic social responsibility for the promotion of sustainability in the university. Such an approach entails, among other aspects, the use of new teaching/learning methodologies, in which teachers cease to be mere transmitters of knowledge. They take on a guiding and companion role for their students in the co-construction of knowledge and its implementation (Gros, 2016). Students, on the other hand, should have a more active role and play a leading role in their own learning process (Gargallo, Jiménez, Martínez, Jiménez, & Pérez, 2017). From the teachers’ perspective, which we have adopted in the present study, a willingness for methodological innovation is one of the competences of the teacher and the professional who seeks to develop quality teaching or work, trying to change themselves for the better. This willingness can be materialized, according to Mas Torelló (2011), in: using different didactic methods to raise students’ awareness towards learning; keeping in touch with the social and professional environment; adequately managing the interactions with the students; and fostering a permanent study attitude that leads to building bridges between research and teaching.

40 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

Regarding the willingness for social responsibility in the university environment, there are widely varied views. There is undoubtedly a large number of teachers who show interest in promoting civic commitment and responsibility. But it is also undeniable that there are teachers who live, in a way, outside of this reality and are more likely to think that engaging in solidarity or service activities is time-consuming, which would mean less time to study (Martínez-Martín, 2010). Fortunately, the latter category is rapidly declining. Life and social reality that we are living in show the need to pay attention to the social dimension in education, and particularly in Higher Education. The professional appointed by the University in 21st century could be characterized (Naval, 2008) as a citizen who participates, is not indifferent (debates the whys and wherefores), focuses on justice, solidarity and the common good, has a lively sense of personal freedom and responsibility. In this context, the Service-Learning methodology is seen as one of the most appropriate to adapt to changes, and new social and educational challenges in the Higher Education environment. There are many definitions of Service-Learning; among all of them, we chose the following: “A pedagogical proposal that addresses the search for specific formulas to involve students in the daily life of communities, neighborhoods, nearby institutions. It is conceptualized within experiential education and is characterized by: a) a prominent role of the students; b) attention to a real need; c) connection with curricular objectives; d) performance of the service project; and e) reflexion” (Naval, García, Puig, & Santos Rego, 2011, p.88). This is an approach that promotes an experiential type of learning in which the teaching staff does not play a prominent role, but those who offer and receive the service. People who participate in the experience are recognized as subjects, and not as objects of the intervention. The individual who receives the service is seen as a core element and not as a simple receiver (Aramburuzabala, 2014). Another defining characteristic of SL is that it meets a real need. It is a holistic, comprehensive, and experiential learning that implies not only the academic institution, but also the community environment. When we reflect and act to meet a real need, our responsibility to learn becomes a contextualized, not fragmented, experience which gives meaning to the subjects that students study.

Revista de Educación, 386. October-December 2019, pp. 37-60 41 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

The learning opportunities offered by real situations allow us to approach university social responsibility in a more complex and holistic manner, widening the choice of improving students’ academic performance. Students engage in processes in which they have to negotiate the concepts and methods exposed in the classroom with the reality and the people involved in the service. The academic contents therefore facilitate the formation of competences for action (Brundiers, Wiek, & Redman, 2010). SL has great potential to breathe life into the contents of the subjects, since community service is designed in connection with the curricular objectives. In a context where we seek to offer opportunities for the learning of social responsibility and curricular competences, the implementation of actual Service-Learning projects is key. The same is applied to most learning situations, such as Mathematics or Music, practice is necessary to absorb and include them into students’ repertoire. When we achieve that the student body feels the need to reflect on and put into action part of curricular contents, the use of reinforcements is not necessary. The SL activities, regardless of the scope of application, share a series of pedagogical conditions (Puig, Gijón, Martín, & Rubio, 2011): a) It is learned from the experience. It is not about solving hypothetical problems or carrying out simulations, but addressing real needs with the intention of improving them. b) It is learned from cooperation. Complex situations are initiated, generating positive interdependence relationships between the participants and with other community members, which would be impossible to assume individually. c) It is learned from reflecting on the action. The reflection is maintained in all phases of the project and is the key element that allows students to appropriate the experience and make it meaningful. d) It is learned with the help provided by the teachers. Students are the main actors of the activity and the teachers become guides or companions in the process, orienting, correcting, stimulating, or organizing the activity. There are numerous studies that support the effectiveness of SL as a teaching/learning methodology. Torío and García-Pérez (2015) observed that, by linking the experience to the concepts and theoretical ideas

42 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables studied by students, the transfer of skills and knowledge to real situations is improved. We also have evidence about the effectiveness of SL in the acquisition of very different competences. Qualitative and meta-analysis-based reviews were conducted, which together comprise hundreds of studies that show positive results of this methodology in academic, personal, social, and professional competences. One could cite, for example, the review conducted by Eyler, Giles, Stenson and Gray (2001), which included 136 qualitative and quantitative studies; the meta-analysis performed by Conway, Amel, and Gerwien (2009) on 103 independent samples; the meta-analysis of Celio, Durlak, and Dymnicki (2011), which summarized the SL effects found in 62 studies that included 11837 unique participants; and the meta-analysis carried out by Yorio and Ye (2012) which focused on 40 studies and 5495 unique participants. The studies conducted by Eyler et al. (2001), and Yorio and Ye (2012) refer exclusively to the effects of SL on higher education, while the two remaining meta-analyses included studies carried out both at university and non-university level. The statistic data used in the meta-analyses show that the greater effect size is associated with academic learning. Moreover, there are positive effects on the learning of civic commitment, social competences or personal attributes such as self-esteem or self-efficacy. Novak, Markey, and Allen (2007) in a meta-analysis conducted with Service-Learning research in higher education, concluded that there was lots of evidence about the advantages of this methodology to understand the subject, learn skills, apply knowledge to practical situations, and analyze complex social problems. There are also research studies which showed that SL, insofar as it is addressed primarily to people and groups who are socially disadvantaged and at risk of exclusion, allows students to become aware of the need for social justice, as well as learning to analyze issues and situations with a critical eye (Herrero & Tapia, 2012; Rubio, Prats, & Gómez, 2013). On the other hand, studies such as those conducted by Folgueiras, Luna, and Puig (2013) or Ibarrola and Artuch (2016) noted that SL contributed to the development of critical thinking and resolution of real problems in a social context, which implies awareness of social responsibility and development of values. These works support the claim that Service- Learning provides an improvement in the academic, personal, social, and professional fields.

Revista de Educación, 386. October-December 2019, pp. 37-60 43 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

This is scientific evidence that, along with the recurrent criticism that the university continues to remain alien to social needs, they are turning SL into a useful revulsive for the management of change in educational institutions. We can say, with little margin for error, that SL is here to stay, if we pay attention to the multiplicity of formal and informal educational fields in which it is being developed, or the quantity and diversity of academic and scientific literature that it generates. Indicators such as the increased number of university and non-university centers that develop it, the quantity and diversity of projects that have been implemented, research on SL in recent years, conferences, meetings, symposia, etc. which periodically mobilize professionals who work with this methodology, the creation of social and academic networks, the amount of articles published in academic journals that address the topic are a clear example of how this methodology and its projection are thriving (García & Cotina, 2015). Service-Learning is being consolidated in the European and Spanish university environment due to the social and economic changes that led the Higher Education institutions to rethink their functions and responsibilities (Rodríguez & Ordoñez, 2015). The “new” social needs demand an adaptation of institutions and services. According to Santos Rego et al. (2015), there are several reasons for proposing the SL methodology in the university classrooms, and they have to do with the improvement of the educational processes, students’ ethical and civic education, as well as with the improvement of the existing linkages between university and society. To this end, Service-Learning should be considered not only as an activity or teaching strategy, but as a part of the university’s training model, as it happens in many Anglo- Saxon universities, and also in some Latin American countries (Santos Rego, 2013). The intention is that, following the mentioned authors, each university incorporates the SL methodology, contributing to its study and dissemination as well as encouraging its use among teachers. One should not leave aside what had been a constant feature in history, that is, in every process of innovation and educational improvement, the teaching staff is a fundamental part. It does not seem feasible to promote at any educational level a teaching practice focused on the achievement of competences, students’ learning or innovation, without affecting in a clear, direct manner the teaching staff and their competences (Mas

44 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

Torelló, 2011). The role of teachers in SL is to organize, motivate, channel, and revitalize experiences. As Naval and Arbués pointed out (2017, p. 199), some of the abilities that teachers should have to promote SL activities among their students, “in addition to the willingness to innovate, are the promotion of dialog, civic responsibility, the ability to communicate with the community, negotiate and reach agreements, to provide a margin of action for students, encouraging them to participate in their own learning process, among others.” But the challenges faced by the university teaching staff for the implementation of SL projects are very important. Some are related to the institutional rigidity as regards the structure of the subjects, the curriculum, its timing, hindering the implementation of the projects, as well as leaving the classroom and the civil responsibility that can be derived from the actions in non-university contexts. Specifically, the purpose in this work is to analyze which arethe variables explaining the introduction of this methodology by a university teacher. Specifically, two objectives were proposed: 1. To study whether certain sociodemographic variables, such as the teaching practice, the engagement with the university’s social commitment, and the interest in innovative teaching practices determine the use of the Service-Learning methodology by the university teaching staff. 2. To establish whether there are significant differences in the predictors according to the teacher’s area of knowledge and the use of SL.

Method

Participants

1903 male and female teachers from 6 Spanish universities participated in the research: University of Córdoba (5.6%), Complutense University of Madrid (18.4%), University of A Coruña (8.9%), University of Navarra (20.8%), University of Santiago de Compostela (19.6%) and University of Valencia.

Revista de Educación, 386. October-December 2019, pp. 37-60 45 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

They belong mainly to the departments of Social and Legal Sciences (34.3%) and Health Sciences (24.3%), and, at a certain percentage distance, to the Experimental Sciences (17%), Arts and Humanities (15%) and, finally, to Technical Studies (9.4%). The different representativeness of the five areas of knowledge is due to the greater presence of certain degree programs in each university, but also to a better precondition of teachers qualified in particular subject areas for being involved in this type of studies. More than half of the teachers are men (52.6%) and 47.4% are women, with an average age of 49.10 years (SD=9.6). They are, mainly, government- employed and statutory teaching staff (associate professors) (69.9%) with an average of 19.21 years (SD = 11.0) of university teaching experience. The subjects that define their teaching style belong to the first (50.7%) and the second (49.3%) cycle of the degree. Most of them (87.7%) teach compulsory subjects to an average of 91.6 students, although in this case the dispersion of answers is logically very large (SD = 74.3), since the minimum is three students and the maximum is 600. Of these, only 27.01% (n=514) are familiar with the Service-Learning methodology, although they do not use it in their subjects (n=408). As shown in Table 1, the teachers of Health Sciences and those of Social and Legal Sciences are those who most admit using this methodology, while those of Experimental Sciences are at the opposite end.

TABLE I. Contingency table of SL use and area of knowledge

SL USE Area of knowledge Yes No n % n % Health Sciences 24 23.3 79 76.7 Experimental sciences 10 13.9 62 86.1 Social and Legal Sciences 51 22 181 78 Technical Studies 5 15.6 27 84.4 Arts and Humanities 16 21.3 59 78.7

Source: Elaborated by the authors.

46 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

Measuring instrument

The Questionnaire on University Professors’ Teaching Practice and Attitude towards Innovation (CUPAIN) was used as a single instrument, designed for an analysis carried out by university teaching staff regarding the organization forms of learning tasks (teaching practice), as well as their attitude towards innovation in university teaching, and more specifically their opinion on the social responsibility of the university. ■ The questionnaire consists of 25 questions formulated in closed categories (22) and of Likert-type questions with five-response choices. The instrument consists of the following sections: ■ Personal and academic-teaching biography questions regarding gender, age, administrative/professional status, university and faculty to which they belong, area of scientific knowledge, degree courses in which they teach, years of teaching experience, positions held at the university, teacher training and membership in any social or civic organizations. ■ Questions on the course which represents their teaching style, placing the teacher in a specific teaching context: type of subject, academic year/cycle in which teaching is provided, number of credits, and enrolled students. ■ Questions related to the Service-Learning methodology to see whether they know of this methodology and whether they use it in their classes. ■ Three Likert-type scales. The first one refers to the Teaching Practice, with 12 items (Cronbach’s α=.82, indicating a high internal consistency). The dimensionality of the scale was determined by means of an Exploratory Factor Analysis (EFA), following the Principal Components extraction method and a Varimax rotation, with the following descriptive statistics: KMO =.88; χ2 (66) = 5025; p < .001. The initial extraction yielded three significant factors that explain 53.60% of the variance: Factor I, external involvement in teaching, includes items that describe activities conducted by the teacher in the course, so that the learning process extends beyond the classrooms, making the most of other resources and opportunities; Factor II, the role of students, is made up of those items that define students’ role in the teaching process; and finally, Factor III, teaching strategies in the classroom, refers to the strategies or methods that the teacher uses in their classes.

Revista de Educación, 386. October-December 2019, pp. 37-60 47 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

Another of the scales, which is unifactorial, refers to the University’s Social Commitment, with 9 items (Cronbach’s α=.85). In the EFA, the initial extraction indicates a significant factor that explains 49.67% of the variance (KMO =.89; χ2 (36) = 5839.68, p < .001). Finally, the third of the scales is the Interest in Innovative Teaching Practices, with 11 items (Cronbach’s α=.85). The initial extraction of EFA (KMO = .89; χ2 (55) = 5839.68, p < .001) revealed two significant factors that explain 52.34% of the variance: Factor I, brings together those items that refer to innovative activities, more focused on the student, whereas Factor II includes the learning activities focused on the interaction with the environment. The study of the design and validation of the instrument can be found in Santos Rego et al. (2017).

Procedure

The questionnaire was applied during the academic year 2015-2016 (October-November) in six universities (five public and one private) and was completed individually online by means of the Survey Monkey software. The instrument was made available to all male and female professors by means of a letter of introduction, since, given the characteristics of this population, and considering the experience of other studies, it was not easy to guarantee a representative sample. Each university adopted a different strategy to reach as many teachers as possible, although in five of them the Vice-Chancellor’s Office was involved, being responsible for the teaching staff and quality. Teachers were sent an institutional email regarding this matter. A letter of introduction had been previously sent to the governing teams.

Data analysis

Given the proposed objectives, we first performed a logistic regression in order to estimate the probability that a university professor uses Service- Learning or not, according to certain sociodemographic variables (gender, age, type of subject taught, number of enrolled students, administrative situation, years of teaching experience, academic year-cycle), the teaching practice, the engagement with the university’s social commitment, and the interest in innovative teaching practices.

48 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

Subsequently, we analyzed the differences in the significant variables of the logistic regression according to the area of knowledge and the use of SL, for which a multivariate analysis of variance (MANOVA) was used 5 X 2 (Area of knowledge X SL use), and a Chi-square test Cycle X SL use, given that the Cycle variable is categorical.

Results

The collinearity statistics were analyzed and tolerance values (T) ranging between .36 and .94 were found; the variance inflation factor (VIF) obtained values between 1.06 and 2.81, indicating there was no collinearity. On the other hand, using the Durbin-Watson test, a value of 1.196 was obtained, indicating independence of the residuals. In the same way, the Hosmer-Lemeshow statistic, with χ2 (514, 8) = 7,38, p = .237, indicated a good fit of the model. Table 2 shows the results of the model including the regression coefficients, the Wald statistic and odds ratio Exp(B). Precisely, the value of Exp(B) indicates how much better or worse are the odds, based on the values assumed by VI. In other words, this is the advantage or disadvantage of owning a certain characteristic, condition or factor.

TABLE II. Results of the logistic regression

Predictors B SE Wald p Exp(B) Gender .46 .23 1.30 .444 .62 Age -.02 .01 .27 .870 .99 Type of subject .40 .28 2.07 .157 1.49 Students enrolled .001 .02 .42 .566 1.00 Administrative situation .20 .04 .85 .359 1.22 Years of experience -.07 .01 .40 .529 .97 Cycle .40 .09 4.70 .030* 1.24 E1-External Involvement .172 .13 1.77 .183 1.19 E1-Teaching Strategies .587 .24 5.93 .015* 1.80 E1-Student’s role .411 .19 4.47 .034* 1.67 E2-Commitment 1.26 .30 17.37 .001*** 3.52 E3-Activ. focused on students .664 .28 5.63 .018* 1.94 E3- Activ. focused on interaction .047 .04 1.50 .220 1.04

*p < .05, **p < .01, ***p < .001 Source: Elaborated by the authors.

Revista de Educación, 386. October-December 2019, pp. 37-60 49 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

As shown in the data, the variable that most predicts the use of the SL methodology by teachers is their engagement with the third mission of the university. When a teacher affirms social commitment of the university, probability of using this methodology is 3.52 times higher than not doing it. This means that SL is linked as a teaching methodology with the teachers’ conviction of the university’s social mission. But the use of SL is also predicted, although to a lesser extent, by the teachers’ interest in innovative activities focused on the student; teaching practice based on strategies such as the use of case studies, teamwork, technologies, in order to encourage student participation and interactivity, or continuous assessment; and the active role of the student in the teaching process. However, most of the sociodemographic variables estimated in the analysis do not predict the use of such methodology. Being male or female, age, teaching an optional or compulsory subject, being a tenured teacher and statutory staff or not, little or great deal of experience or the number of students enrolled do not determine the use of this methodology. In fact, only one was significant. More specifically, the cycle-academic year in which the professor teaches the subject that, in their opinion, represents their teaching style is a variable predictive of the SL use. Thus, for each teacher who uses this methodology in the first cycle (1st and 2nd year) there are 1.24 who do it in the second cycle (3rd and 4th year). In short, the teachers who teach in the second university cycle and whose orientation is not focused on teaching (reproductive model, focused on the teacher, and the transfer of information) but on learning (constructivist model, focused on the student, facilitating learning) (Gargallo, 2008) most likely employ SL. One should remember that the Bologna Declaration implies a clear methodological change in the teaching-learning process, with active and participatory learning- centered methodologies, and a change in teachers’ role. This tendency shifts, therefore, the focus of interest from the unidirectional information to the interaction, from knowing to knowing and acquiring awareness, from the classroom to reality, from exposure to problem solving, and from individualism to cooperation (Martínez-Martín, 2006).

50 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

TABLE III. Classification table

Predicted Observed Correct percentage No SL use SL use No SL use 400 6 98.5 SL use 101 5 4.7 Total percentage 79.1

Source: Elaborated by the authors.

The classification table shows that 79.1% of the participants are correctly classified, but in the case of those who do not use SL,the specificity is higher (98.5%). Next, we analyzed the differences in these five variables using a multivariate analysis of variance (MANOVA) 5 X 2 (Area of knowledge X SL use). The first thing to note is that there were statistically significant differences in the ‘Teaching strategies in the classroom’ variable (Factor III of the Teaching Practice Scale - Scale 1) depending on whether SL is used or not, F(1,504) = 5.04, p = .025, η2 = .010, 1-β = .611. Teachers who use this methodology obtained a higher mean in this variable (Mno

= 3.90, Myes = 4.12); that is, they encourage the active participation of students in the classroom, promote activities to enhance critical thinking, use students’ own experiences and make sure there is a good climate of interpersonal relationships. However, in the ‘Student’s Role’ variable (Factor II of the Teaching Practice Scale - Scale 1) no statistically significant differences were established according to whether the teacher chose this methodology or not. We also found statistically significant differences in the ‘University’s Social Commitment’ variable (Scale 2) depending on whether this methodology is used or not, F(1,504) = 20.30, p < .001, η2 = .039, 1-β = .994. The teachers who have already included SL are those who most affirm the university’s social commitment (Mno = 4.36, Msi = 4.68). Likewise, there were statistically significant differences in the variable Learning Activities focused on the student (Factor I of the Learning Activities Scale - Scale 3) if SL is used or not, F(1,504) = 5.17, p = .023, η2 =

Revista de Educación, 386. October-December 2019, pp. 37-60 51 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

.011, 1-β = .622. The university teachers accustomed to this methodology are the ones who are most interested in activities of methodology updating, which involve problem solving, promote students’ participation, autonomous and permanent learning, and develop students’ critical

thinking (Mno = 4.52, Myes = 4.67). On the other hand, a significant relationship was also obtained between using this methodology or not by the cycle in which the subject is taught (see Table 4).

TABLE IV. Contingency table of Cycle X SL use

SL USE Total No Yes No. of students 213 46 259 1st Expected frequency 205.6 53.4 259 % within CYCLE 82.2% 17.8% 100% Cycle No. of students 195 60 255 2nd Expected frequency 202.4 52.6 255 % within CYCLE 76.5% 23.5% 100% No. of students 408 106 514 Total Expected frequency 408 106 514 % within CYCLE 79.4% 20.6% 100%

Source: Elaborated by the authors.

As shown in the Table, the teachers who teach subjects in the second cycle are those using SL [χ2 (514, 1)= 3.61, p = .047]. In addition, there is a statistically significant interaction between the ‘Area of knowledge’ and ‘SL use’ variables in the dependent variable of ‘University’s Social Commitment’, F (1,504) = 2.34, p = .049, η2 = .018, 1-β = .673 (see Figure 1). After the corresponding post hoc contrasts, it was observed that: ■ No statistically significant differences were found between using SL or not in the variable of ‘Commitment in the Degree Programs of Health Sciences and Social and Legal Sciences’. That is, teachers

52 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

belonging to these areas, being those who most use it in their subjects, obtained similar scores in social commitment, independently of the methodology, although the average is higher for those who used SL. ■ Statistically significant differences were observed in the following

degree programs: Experimental Sciences (Mno-Myes = -.44, p=.040),

Technical Studies (Mno-Myes = -.48, p=.037) and Arts and Humanities

(Mno-Myes = -.45, p=.040). In all three cases, the means were higher when the teaching staff used SL, indicating that the teachers in these areas who incorporated SL into their work are firmer regarding the third mission of the university. In any case, we observed that among the university teachers who did not use SL, the most convinced of the social commitment of the university are those who teach Health Sciences and Social and Legal Sciences, which could be explained by a stronger link of the curriculum of these degrees with a service to the society; on the other hand, they obtained the lowest score in this variable among those who used this methodology, those belonging to Technical Studies and Arts and Humanities being worth mentioning (see Graph I). That is, among the teachers that use SL, the most convinced of the University’s social commitment seem to be the teachers of Technical Studies and Arts and Humanities.

GRAPH I. Interaction Area of Knowledge X SL use in the ‘Social Commitment’ variable

Source: Elaborated by the authors.

Revista de Educación, 386. October-December 2019, pp. 37-60 53 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

Discussion and Conclusions

Regarding the first objective, we have to conclude that of all the analyzed variables, the one that better predicts the SL use by the university teaching staff is their engagement with the university’s social commitment, understood as the third mission, beyond teaching and research. On the same vein, Holland (1999) analyzed the reasons why university teachers were oriented towards activities that implied a social commitment to highlight how these professionals, with a high level of training, saw themselves with the responsibility of applying their knowledge to improve society; but this author also pointed out the results of the students’ learning or the prestige of the university centers. Finally, Service-Learning was targeted as a way that can boost teachers’ social commitment. The teachers of our study are also characterized by their interest in innovative activities focused on the student, by a teaching practice defined by more constructivist strategies and, in line with the above, the consideration of the active role of the student in the teaching process. On the contrary, of the seven sociodemographic variables studied, only the ‘academic year-cycle in which teaching is provided’ is a good predictor of the use of this methodology. Thus, the probability that a university teacher uses SL increases in the last two years of the degree program, which agrees with the idea stated by Deeley (2015) that Service-Learning was more appropriate to be used in advanced academic courses of the university, because in this way a higher level of preparation can be guaranteed for the students. Hammond (1994), using a sample of 163 teachers from Michigan State University, studied the profile of those who employed Service-Learning. The author found that teachers adopted this methodology for curricular reasons, especially because of the relevance that SL provided to the course material. This is an effective means of experiential education and it encourages self-directed learning, that is, students are more autonomous, are able to learn by themselves, and solve their own problems. On the other hand, it was found that teachers who used SL in their teaching practice (teaching strategies) were different from those who did not, as well as in their affirmation regarding the university’s social commitment, in their interest for innovative teaching activities (focused on the student), and the cycle (2nd) in which teaching is provided. However,

54 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables although there is no difference between teachers in the areas of Social and Legal Sciences and Health Sciences who use this methodology or not when affirming the social commitment of the institution, we did find differences in the other areas. The teachers belonging to Experimental Sciences, Technical Studies, and Arts and Humanities that used SL expressed strongly in favor of that third mission of the university. In the same way, Wade and Demb (2009) analyzed the variables that affected teachers’ social commitment in order to group them into four large dimensions: institutional, professional, personal and commitment of the faculty. In the latter, Service-Learning is pointed out, while among the professional variables, the area or discipline of the teacher stands out. Those teaching Social and Health Sciences are more likely to participate in service activities. To conclude, we can state that Service-Learning is an excellent framework that allows universities to assume their social mission not only institutionally (strategic plans, social responsibility reports), but also on the ground, in a mutual dialog with the society to achieve a more democratic and just world. But to this end, it is undoubtedly necessary that within the training plans of the teaching staff that have been activated in the Spanish universities, a line of work oriented towards acquiring the theoretical foundations and the practical tools should be maximized, allowing teachers to develop the social function through teaching (Alvarez, Martínez, González, & Buenestado, 2017). This must go hand in hand with the firm commitment of the university institution for innovation in the classroom. It is clear from our study that teachers need training and encouragement to develop SL projects that could constitute good practices of university action. One should keep in mind that only between 13.9 and 23.3% of the teachers in the sample, depending on their area of knowledge, use SL. The introduction of SL in the university dynamics does not only imply benefits for the students, but also contributes to the exercise of a more active citizenship, promotes a socially committed academic culture between the teaching staff and the universities, establishes close connections with the community, and enhances social justice (Butin, 2003). However, our research has limitations that we should address in future work. Besides the analysis of the variables that explain the fact that the university teaching staff uses SL, it would also be interesting to include,

Revista de Educación, 386. October-December 2019, pp. 37-60 55 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

in further research on the subject, other variables that may influence this choice. This refers to sociodemographic variables of the teacher, such as participation in social organizations, and others that have to do with institutional policy. In short, if what we aspire to is a sustainable development of SL in our campuses, favoring innovation and the development of learning processes, it is necessary to assess the projects that these teachers are developing, obtaining evidence which would allows us to defend the potential of this methodology for all areas of knowledge.

References

Alvarez, J.L., Martínez, M.J., González, H., & Buenestado, M. (2017). El aprendizaje-servicio en la formación del profesorado de las universidades españolas [Service-Learning in the training of Spanish university teaching staff]. Revista Española de Pedagogía, 75(267), 199- 217. doi: 10.22550/REP75-2-2017-02 Aramburuzabala, P. (2014). Aprendizaje-servicio. Ciudadanía activa, justicia social y aprendizaje [Active citizenship, social justice and learning]. In V. Ballesteros Alarcón (Coord.), Implicaciones de la educación y el voluntariado en la formación de una ciudadanía activa. Perspectiva internacional (pp 33-48). Granada: GEU. Brundiers, K., Wiek, A., & Redman, C. L. (2010). Real-world learning opportunities in sustainability: from classroom into the real world. International Journal of Sustainability in Higher Education, 11(4), 308-327. doi: 10.1108/14676371011077540. Butin, D. (2003). Of what use is it? Multiple conceptualizations of service learning within education. Teachers College Record, 105(9), 1674-1692. Celio, C. I., Durlak, J., & Dymnicki, A. (2011). A meta-analysis of the impact of service learning on students. Journal of Experiential Education, 34(2), 164-181. doi: 10.5193/JEE34.2.164 Conway, J. M., Amel, E. L., & Gerwien, D. P. (2009). Teaching and learning in the social context: A meta-analysis of service learning’s effects on academic, personal, social, and citizenship outcomes. Teaching of Psychology, 36(4), 233-245.

56 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

CRUE (2015). Institucionalización del Aprendizaje-Servicio como estrategia docente dentro del marco de la Responsabilidad Social Universitaria para la promoción de la Sostenibilidad en la Universidad [Institutionalization of Service-Learning as a teaching strategy within the framework of the Academic Social Responsibility for the Promotion of Sustainability in the University]. Recovered from https://www.crue.org/Documentos compartidos/Recomendaciones y criterios tecnicos/2. APROBADA INSTITUCIONALIZACION ApS.pdf. Deeley, S. (2015). Critical perspectives on Service-Learning in higher education. Basingstoke: Palgrave Macmillan. Eyler, J., Giles, D. E. Jr., Stenson, C. M., & Gray C. J. (2001). At a glance: What we know about the effects of Service-Learning on College students, faculty, institutions and communities, 1993-2000 (3rd edition). New York, NY: Corporation for National Service. Recovered from https:// www.mnsu.edu/cetl/academicservicelearning/Service-Learning.pdf. Folgueiras, P., Luna, E., & Puig, G. (2013). Aprendizaje y servicio: estudio del grado de satisfacción de estudiantes universitarios [Service and learning: study of the degree of satisfaction of university students]. Revista de Educación, 362, 159-185. doi: 10-4438/1988-592X-RE-2011- 362-157. García, M., & Cotina, J. M. (2015). El Aprendizaje-Servicio en la formación inicial del profesorado: de las prácticas educativas críticas a la institucionalización curricular [Service-Learning in initial teacher education: from critical educational practices to curricular institutionalization]. Profesorado: Revista de curriculum y formación del profesorado, 19, 8-25. Gargallo, B. (2008). Estilos de docencia y evaluación de los profesores universitarios y su influencia sobre los modos de aprender de sus estudiantes [Teaching and assessment styles of university teaching staff and the influence on their students’ ways of learning]. Revista Española de Pedagogía, 241, 425-446. Gargallo, B., Jiménez, M.A., Martínez, N., Jiménez, J.A., & Pérez, C. (2017). Métodos centrados en el aprendizaje, implicación del alumno y percepción del contexto de aprendizaje en estudiantes universitarios [Methods focused on learning, student involvement and perception of the learning context in university students]. Educación XX1, 20(2), 161-187, doi: 10.5944/educXX1.15153

Revista de Educación, 386. October-December 2019, pp. 37-60 57 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

Gros, B. (2016). Tecnologías digitales e innovación educativa: retos de una relación inevitable [Digital technologies and educational innovation: challenges of an inevitable relationship]. In J. M. Mominó, & C. Sigalés (Coords.), El impacto de las TIC en educación: más allá de las promesas (pp. 157-176). Barcelona: UOC ediciones. Hammond, C. (1994). Integrating service and academic study: faculty motivation and satisfaction in Michigan higher education. Michigan Journal of Community Service Learning, 1(1), 21-28. Herrero, M. A., & Tapia, M. N. (Comps.) (2012). Actas de la II Jornada de investigadores sobre aprendizaje-servicio [Proceedings of the 2nd Conference of researchers on Service-Learning]. Buenos Aires: CLAYSS-Red Iberoamericana de Aprendizaje-Servicio. Recovered from http://www.clayss.org/JIAS/II_jias/Libro_IIJIA-S_COMPLETO.pdf Holland, B. (1999). Factors and strategies that influence faculty involvement in public service. Journal of Public Service and Outreach, 4(1), 37-43. Ibarrola, S. y Artuch, R. (2016). La docencia en la universidad y el compromiso social y educativo [Teaching in the university and the social and educational commitment]. Contextos Educativos, 19, 105- 120. doi: 10.18172/con.2763. Martínez-Martín, M. (2006). Formación para la ciudadanía y educación superior [Training for citizenship and higher education]. Revista Iberoamericana de Educación, 42, 85-102. Martínez-Martín, M. (2010). Aprendizaje Servicio y Responsabilidad Social de las universidades [Service-Learning and Social Responsibility of universities]. Barcelona: Octaedro. Martínez-Martín, M. (2016). Responsabilidad social de la universidad en el marco de la sociedad abierta [Social responsibility of the university within the framework of the open society]. In M. A. Santos Rego (Ed.), Sociedad del conocimiento. Aprendizaje e innovación en la universidad (pp. 139-153). Madrid: Biblioteca Nueva. Mas Torelló, O. (2011). El profesor universitario: sus competencias y formación [The university teacher: competences and training]. Profesorado. Revista de currículum y formación del profesorado, 15(3), 195-211. Ministry of Education (2011). Estrategia Universidad 2015. Contribución de las universidades al progreso socioeconómico español [University Strategy 2015. Contribution of universities to Spanish socio-economic progress 2010-2015]. Madrid: Ministry of Education.

58 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

Naval, C. (2008). Universidad y conciencia cívica. Algunas experiencias fructíferas: Service-Learning y campus compact [University and civic awareness. A few successful experiences: Service-Learning and Compact Campus]. Revista Sembrando Ideas, 2. Recovered from http:// dadun.unav.edu/bitstream/10171/20567/1/54.pdf. Naval, C., & Arbués, E. (2017). El aprendizaje-servicio en la educación superior: las competencias profesionales [Service-Learning in higher education: professional competences]. In J. A. Ibáñez Martín, & J. L. Fuentes (Eds.), Educación y Capacidades: Hacia un nuevo enfoque de desarrollo humano (pp. 189-207). Madrid: Dykinson. Naval, C., & Arbués, E. (Eds.) (2018). Hacer la Universidad en el espacio social [Constructing the university in the social space]. Pamplona: EUNSA. Naval, C., García, R., Puig, J.Mª, & Santos Rego, M.A. (2011). La formación ético-cívica y el compromiso social de los estudiantes universitarios [Ethical-civic training and social commitment of university students]. Encounters on Education, 12, 77-91. Novak, J. M., Markey, V., & Allen, M. (2007). Evaluating Cognitive Outcomes of Service Learning in Higher Education: A Meta- Analysis. Communication Research Reports, 24(2), 149-157. doi: 10.1080/08824090701304881. Piqué, B., & Forés, A. (2013). Propuestas metodológicas [Methodological proposals]. In M. A. Escofet & G. Bautista (Eds.), Enseñar y aprender en la universidad. Claves y retos para la mejora (pp. 171-202). Barcelona: Octaedro-ICE. Puig, J. M., Gijón, M., Martín, X., & Rubio, L. (2011). Aprendizaje Servicio y Educación para la Ciudadanía [Service-Learning and Education for Citizenship]. Revista de Educación, número extraordinario, 45-67. Rodríguez, M., & Ordoñez, R. (2015). Una experiencia de Aprendizaje- Servicio en Comunidades de Aprendizaje [A Service-Learning experience in Learning Communities]. Profesorado: Revista de curriculum y formación del profesorado, 19, 314-333. Rubio, L., Prats, E., & Gómez, L. (Coords.) (2013). Universidad y Sociedad. Experiencias de Aprendizaje-Servicio en la Universidad [University and Society. Service-Learning Experiences in the University]. Barcelona: Universitat de Barcelona i Institut de Ciències de l’Educació. Santos Rego, M. A. (2013). ¿Para cuándo las universidades en la agenda de una democracia fuerte? educación, aprendizaje y compromiso

Revista de Educación, 386. October-December 2019, pp. 37-60 59 Received: 25-07-2018 Accepted: 24-05-2019 Lorenzo Moledo, M. M., Ferraces Otero, M.J., Pérez Pérez C. Naval Durán, C. University teaching staff facing Service-Learning: explanatory variables

cívico en Norteamérica [When will universities be on the agenda of a strong democracy? Education, learning and civic engagement in North America]. Revista de Educación, 361, 565-590 Santos Rego, M.A., Jover, G., Naval, C., Álvarez, J.L., Vázquez V., & Sotelino, A. (2017). Diseño y validación de un cuestionario sobre práctica docente y actitud del profesorado universitario hacia la innovación (CUPAIN) [Design and validation of a questionnaire on university professors’ teaching practice and attitude towards innovation (CUPAIN)]. Educación XX1, 20(2), 39-71, doi: 10.5944/educXX1.17806 Santos Rego, M.A., Sotelino, A., & Lorenzo, M. (2015). Aprendizaje- servicio y misión cívica de la universidad: una propuesta de desarrollo [Service-Learning and the University’s civic mission: a proposal for development]. Barcelona: Octaedro. Torío, S., & García-Pérez, O. (2015). Aprendizaje-Servicio, estrategia para la participación social en la Universidad [Service-Learning, a strategy for social participation in the University]. In G. Pérez Serrano (Coord.), Pedagogía social, universidad y sociedad (pp. 267-276). Madrid: Universidad Nacional de Educación a Distancia. Wade, A., & Demb, A. (2009). A conceptual model to explore faculty community engagement. Michigan Journal of Community Service Learning, 15, 5-16. http://hdl.handle.net/2027/spo.3239521.0015.201 Yáñez-Aldecoa, C., Okada, A., & Palau, R. (2015). New learning scenarios for the 21st century related to Education, Culture and Technology. Revista de universidad y sociedad del conocimiento, 12(2), 87-102. doi: 10.7238/rusc.v12i2.2454 Yorio, P. L., & Ye, F. (2012). A meta-analysis on the effects of Service- Learning on the social, personal, and cognitive outcomes of learning. Academy of Management Learning & Education, 11(1), 9-27. doi: 10.5465/amle.2010.0072

Contact address: María del Mar Lorenzo Moledo. Universidade de Santiago de Compostela, Facultad de Ciencias de la Educación. Campus Vida, 15782, Santiago de Compostela. E-mail: [email protected]

60 Revista de Educación, 386. October-December 2019, pp. 37-60 Received: 25-07-2018 Accepted: 24-05-2019 Bayesian analysis of barriers towards entrepreneurship at university

Análisis bayesiano de barreras al emprendimiento en la universidad

DOI: 10.4438/1988-592X-RE-2019-386-427

Ana María Ruíz-Ruano García UCAM Universidad Católica de Murcia María del Pilar Casado Belmonte Universidad de Almería Jorge López Puga UCAM Universidad Católica de Murcia

Abstract The creation of small and medium sized businesses is one of the employment alternatives available to university graduates in any area of knowledge, as well as a mechanism through which university teaching staff can transfer useful or relevant knowledge to society. Given that entrepreneurship is considered a key element to boost societies and optimize the economic system, promoting or facilitating the creation of companies at university level should be considered desirable. There are multiple strategies designed to foster entrepreneurship and some of the most relevant, both theoretical and practical, are based on models of attitudinal changes. Recent theoretical models in the study of entrepreneurship highlight that the perception of obstacles by potential entrepreneurs can explain their entrepreneurial creation and intention. This work uses inference and Bayesian networks to model the impact of different potential obstacles on the intention and creation of companies among university teaching staff and students. A sample of 2671 lecturers and 1224 university students (1744 were women and 1917 were men) completed a survey to identify the perceived potential obstacles to business creation. The R bnlearn statistical package was used to estimate structural models of the Bayesian network that modelled

Revista de Educación, 386. October-December 2019, pp. 61-84 61 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

creation and entrepreneurial intention based on perceived obstacles. The results show that teaching staff and students perceive different potential barriers to entrepreneurship. While teaching staff created companies despite certain perceived obstacles, the intention and creation of companies is inhibited among students depending on the perceived obstacles. These results can be used to improve university entrepreneurship empowerment programs and to optimize the chances of success in potential entrepreneurs setting out from university.

Key words: academic entrepreneurship, entrepreneurial intention, obstacles, university, Bayesian inference.

Resumen La creación de pequeñas y medianas empresas es una de las alternativas laborales disponibles para los egresados universitarios en cualquier área de conocimiento, así como un mecanismo por medio del cual el profesorado universitario puede transferir conocimiento útil o relevante a la sociedad. Dado que el emprendimiento es considerado como un elemento clave para dinamizar las sociedades y optimizar el sistema económico, el fomento o la facilitación de la creación de empresas a nivel universitario puede considerarse deseable. Existen múltiples estrategias diseñadas para auspiciar el emprendimiento y algunas de las más relevantes, tanto teórica como prácticamente, se basan en los modelos de cambios de actitudes. Los recientes modelos teóricos en el estudio del emprendimiento señalan que la percepción de obstáculos por parte de los emprendedores potenciales puede explicar la creación e intención emprendedora. Este trabajo utiliza la inferencia y las redes bayesianas para modelar el impacto que tienen diferentes obstáculos potenciales sobre la intención y creación de empresas entre profesorado y alumnado universitario. Una muestra integrada por 2671 profesores y 1224 alumnos universitarios (1744 fueron mujeres y 1917 fueron hombres) cumplimentó una encuesta destinada a identificar los obstáculos potenciales percibidos frente a la creación de empresas. Se utilizó el paquete estadístico bnlearn de R para estimar los modelos estructurales de red bayesiana que modelaban la creación y la intención emprendedora en función de los obstáculos percibidos. Los resultados muestran que profesorado y alumnado perciben diferencialmente las barreras potenciales al emprendimiento. Mientras que el profesorado crea empresas a pesar de ciertos obstáculos percibidos, la intención y la creación de empresas se inhibe en el alumnado en función de los correspondientes obstáculos percibidos. Estos resultados pueden ser utilizados para mejorar los programas de potenciación del emprendimiento universitario y para optimizar las probabilidades de éxito en los emprendedores potenciales originados en la universidad.

Palabras clave: emprendimiento académico, intención emprendedora, obstáculos, universidad, inferencia bayesiana.

62 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Introducción

The study of entrepreneurship is not a new subject, however, it is an issue that continues to arouse interest today. From the university context, there is an increasing demand for a change from the traditional university to the entrepreneurial university, achieving greater prominence in economic and social areas (e.g., Abbott, Zastrow, Gibney, & Nordling, 2014; Nature, 2014; Etzkowitz, Webster, Gebhardt, & Cantisano, 2000; Glassman et al., 2003; Kuratko, 2005; Rodeiro, Fernández, Rodríguez, & Otero, 2012; Toledano, 2006; Watson-Capps & Cech, 2014). This change should aim to increase the employability of university graduates (e.g., Haller & Welch, 2014; Jones, Macpherson, & Woollard, 2008; Mars, 2009). In this sense, one of the possibilities that university students may consider will increase their chances of entering the labour market is by considering entrepreneurship as a possible way to employment. The aim of this paper is to present the exploratory-correlational analysis carried out with Bayesian statistics, in which it is intended to analyse the impact that certain potential obstacles have on university teaching staff and students on the intention and creation of companies. In this way, differential patterns can be identified in relation to entrepreneurship among students and university teaching staff, thus helping to design teaching strategies so that they emphasize the obstacles identified as being most important, as well as the fact that they can be perceived as opportunities (Nabi, Walmsley, Liñán, Akhtar, & Neame, 2016). Employability as a result of university education The employability of university graduates is a question that is not without controversy, especially if the phenomenon known as brain drain is addressed (Santos, 2013, Santos & Muñoz-Rodríguez, 2015, 2017). However, this phenomenon of mobility, far from being considered as something negative, can also be understood as positive. In fact, mobility in many cases is generated by globalization and the demand for specialized profiles in other parts of the world. Santos and Muñoz-Rodríguez (2015) interpret the mobility of qualified personnel, in this case of university graduates, as self-employment measures. In addition, these new profiles of workers have the ability to keep the economy active (Sánchez, 2011, Shapero, 1985, Van Praag & Versloot, 2007). In this sense, university curricula have been adjusted to focus higher education on the demands of the business context in accordance with the

Revista de Educación, 386. October-December 2019, pp. 61-84 63 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

European Higher Education Framework and the Bologna plan (Santos & Muñoz-Rodríguez, 2017). Toledano (2006) points out that the university plays a fundamental role in promoting entrepreneurial initiative among its students and claims that higher education institutions should be the ones that take on the role of business activity stimulators. This way, it is necessary for the Spanish university to undertake a change towards the entrepreneurial university, as reflected in the preamble of the current Organic Law on Universities (Jefatura del Estado, 2001). However, this regulation is focused on the transfer of knowledge by lecturers, in terms of providing companies with scientific knowledge developed in the university, it does not echo the importance of providing students with the ability to set up their own businesses. The Organic Law on Education (Jefatura del Estado, 2006), in an effort to adapt education to the current social and economic reality, included the development of the entrepreneurial spirit as a fundamental competence for student development. However, although this effort to highlight the importance of entrepreneurship is indicated in primary education, secondary education, baccalaureate and vocational training, with the inclusion of specific subjects, it is not made in university higher education (Jefatura del Estado, 2013). Similarly, there are many initiatives carried out in different parts of the Spanish territory to promote entrepreneurship at different educational levels (e.g., Bernal, 2014; Marina, 2010). However, the Organic Law on Universities does not explicitly mention entrepreneurship. For this reason, Toledano (2006) demands this entrepreneurial spirit be a generic objective of the University so that students can take advantage of the knowledge acquired during their training. However, the university encourages entrepreneurship through other means such as knowledge transfer, international mobility or other initiatives at the national level (Marina, 2010). The Government of Spain, aware of the importance of entrepreneurship in the university, launched a program in 2013 called Entrepreneurship Initiative at University as a joint initiative of the Ministry of Industry, Energy and Tourism, and the Ministry of Education, Culture and Sports (Ministry of Industry, Energy and Tourism, 2014, 2015). The objective of this program was “to promote entrepreneurial and business activity among university students as a way out of economic and employment difficulties” (p.1). Consequently, there are different studies that have tried to analyse the effects of the implementation of programs to promote

64 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university entrepreneurship, but their results are, at least, contradictory since in some cases positive effects were obtained while in others they were not (Barba-Sánchez & Atienza-Sahuquillo, 2018; Block, Hoogerheide, & Thurik, 2013; Dohse & Walter, 2012; Nabi et al., 2016; Oosterbeek, van Praag, & Jsselstein, 2010; Souitaris, Zerbinati y Al-Lahan, 2007; von Graevenitz, Harhoff, & Weber, 2010).

Entrepreneurship at university and creation of companies

Despite inconclusive results on the impact of education on entrepreneurship, there is a general consensus on the inclusion of entrepreneurship at university, it being considered a key aspect in innovation, productivity and competitiveness (Plaschka & Welsch, 1990). Accordingly, efforts are being made worldwide to implement various actions in higher education in order to promote such entrepreneurship. Among others, the inclusion of a subject related to qualifications in business creation (e.g., Kuratko, 2005; Sardeshmukh & Smith-Nelson, 2011), the creation of support units for entrepreneurship (Crum & Chen, 2015) or competitions for entrepreneurial ideas (Henry, Hill, & Leitch, 2005). Traditionally, actions linked to university to promote entrepreneurship have supported the creation of companies, research contracts and the development of patents (Rodeiro et al., 2012). In order to do this, Spanish universities have Employment Services, which provide advice for students to help them set themselves up as entrepreneurs. The existence of business incubators at universities is also becoming a reality in order to try to foster this line of entrepreneurship. Or the creation of Research Results Transfer Offices, the so-called OTRI in charge of energizing the contact between the university and companies. Another methodology to attract companies to university campuses, as a way to pave the way for students to the entrepreneurial environment is co-location (Watson- Capps & Cech, 2014). For its part, spin-offs are the best-known companies generated within the university sphere (Patzelt & Shepherd, 2009). Another initiative is the licensing of technology or technology transfer (Watson-Capps & Cech, 2014), with the purpose of transferring scientific advances for exploitation at the company level.

Revista de Educación, 386. October-December 2019, pp. 61-84 65 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Explanatory models of entrepreneurship

The prevailing models to explain entrepreneurship from a psychological perspective are those based on attitudes (Robinson, Stimpson, Huefner, & Hunt, 1991). However, these models do not contemplate barriers to entrepreneurship as being important variables in the entrepreneurial process. That is why Lüthje & Franke (2003) proposed a new model to explain the entrepreneurial intention, in which in addition to contemplating classic aspects such as attitudes or personality traits, they added the perception of both barriers and supports with a direct influence on entrepreneurial intention. More recently, Rueda, Fernández-Laviada & Herrero (2014) proposed another model in which they combine the theory of planned behaviour (Ajzen & Fishbein, 2015) with the perception of advantages and disadvantages towards entrepreneurship. Other authors who have tried to integrate the different models to explain the entrepreneurial intention have also included the perception of barriers and supports (e.g. López, 2009, Ruiz-Ruano, 2015). Although barriers or obstacles to entrepreneurship are not presented as the variable that most influences entrepreneurial intention from an attitudinal perspective (Ruiz-Ruano, 2015), some authors maintain that it is their perception which is the determining factor for an intention to turn into action (Armitage & Conner, 2001). In fact, different studies have been carried out in which an attempt has been made to categorize the perceived barriers to entrepreneurship, in an attempt to help potential entrepreneurs to develop their projects (e.g., Borbera, Lekovic, & Berber, 2014; Deh, Yeboah, & Agyemang, 2013; Iakloveva, Kolvereid, Gorgievsky, & Sørhang, 2014; Rueda, Sánchez, Herrero, Blanco, & Fernández- Laviada, 2013). For example, economic or financial resources have been considered in many previous studies as obstacles to entrepreneurship and, in other cases, as facilitators of the same. For example, Nabi et al. (2006) point out economic risk as a reason not to take action on an idea; however, Rodeiro et al. (2012) point to the availability of resources as the determining factor in the realization of patents to be undertaken.

Statistical inference and Bayesian networks

For a wide spectrum of knowledge areas, the main statistical data analysis tools are based on the Null Hypothesis Significance Test (NHST). However,

66 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university the exclusive use of these statistical tools in scientific research has been criticized both in the past and in the present (e.g., Bakan, 1966; Cohen, 1994; Halsey, Currant-Everett, Vowler, & Drummond, 2015; Orlitzky, 2012; Rosnow & Rosenthal, 1989; Stern, 2016; Wasserstein & Lazar, 2016). Some of these criticisms revolve around the use and interpretation of the p-value of a test, as well as in relation to the typology of the inference made. For example, while the classical approach to statistical inference is centred on the credibility of the sample data given the hypotheses, P(D|H); the Bayesian inference (e.g., Anscombe, 1961; Bolstad, 2007; Jeon and De Boeck, 2017; Van Zyl, 2018) aims to approach what is really of interest for researchers, the probability of the hypotheses given the sample data, P(H|D). The Bayes Factor (FB) can be considered as one of the basic tools provided by Bayesian statistical inference (Kass & Raftery, 1995). In this context, the Bayes Factor favourable to the null hypothesis (FB01) or to the alternative (FB10) are being used more and more as statistics that justify or support statistical decision making (Jeon & De Boeck, 2017). The statistic is interpreted as the statistical evidence that exists in favour of the null hypothesis, or vice versa, given the sample data versus the evidence that exists concerning the alternative hypothesis, or vice versa. Bayes Factors equal to one indicate that both hypotheses are equally likely given the sample data while values greater or less than this value indicate that there is favourable evidence towards one hypothesis or another. Further important progress deriving from Bayesian inference are Bayesian networks as multivariate tools for statistical modelling (eg, Puga, Krzywinski, & Altman, 2015). These are graphical probabilistic models that represent the set of relationships of dependence and conditional dependence that are established between the variables of a problem (Cowell, Dawid, Lauritzen, & Spiegelhalter, 1999). They have the advantage of representing in unison the qualitative and quantitative dimension of a problem, being especially useful in decision-making contexts under the perspective of Bayesian inference (Edwards & Fasolo, 2001).

Objectives

The objective of this paper is to study the perception of barriers to entrepreneurship in the university context. It will be analysed differentially

Revista de Educación, 386. October-December 2019, pp. 61-84 67 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

dealing with two groups of agents clearly differentiated in the university: the teaching staff and students. In addition, they will be analysed both from an intentional perspective (potential entrepreneurship), and from a consolidated perspective (actually executed entrepreneurship). Thus, we can establish if there are striking differences between the perception of obstacles between real entrepreneurs and potential entrepreneurs among both teaching staff and university students. In this sense, we hope that the perception of barriers or obstacles is greater among students than among the teaching staff. It is also expected that there will be a greater number of obstacles related to entrepreneurial intention than to the fact of having created a company. That is to say, that the people who have the intention of creating a company will perceive more obstacles than those perceived by the people who have really created their company. This study is important because it tries to identify differential patterns that allow the entrepreneurial spirit to be transmitted or fostered among students and the university teaching staff. This can help to design better strategies, and to try to see that what can a priori be understood as a difficulty can also be understood as an opportunity (Nabi et al., 2016). Therefore, we believe that this work has a dual purpose: a) from the applied point of view it can help in the improvement of the management of public or private policies aimed at improving entrepreneurship, and b) from the scientific point of view it could be considered as an exploratory study (Altman & Krzywinski, 2018; Nosek, Ebersole, DeHaven, & Mellor, 2018) that allows entrepreneurship to be understood in the academic context with the corresponding utility that it would have faced with the generation of new theories in this regard.

Method

Sample

To carry out the study, a sample of 3,895 people was used, of which 1,224 (31.4%) were university students from the University of Almeria and the remaining 2,671 (68.6%) were university teaching staff from public universities in Andalusia. The proportion of women (n = 1,745, 44.8%) and men (n = 1,917, 49.2%) were very similar (6% did not respond),

68 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university while the ages were between 17 and 77 years (M = 37.17, SD = 13.24). The average age of the group of students was 22.46 (SD = 4.47) while that of the group of teaching staff was 44.43 (SD = 9.61). Although the proportion of people who said they had taken part in the creation of a company (n = 869, 22.3%) or intended to create one in the short term (n = 1,030, 26.4%) did not exceed one third of the participants; most of them considered the creation of companies desirable (n = 1,925, 64.7%). On the other hand, only 960 people (24.6%) indicated that the creation of companies at the present time was feasible. Most of the participants indicated that the area of knowledge in which they developed their academic activity fell within the scope of the Social and Legal Sciences (n = 1,680, 43.1%), in the field of Engineering and Architecture (n = 854, 21.9%) or in Science (n = 524, 13.5). On the other hand, the areas that had fewest participants were the Health Sciences (n = 218, 5.6%) and the field of Art and Humanities n( = 297, 7.6%).

Instruments

Data collection was carried out by means of a printed questionnaire (for students) and by means of an electronic form (for the teaching staff). In addition to sociodemographic information, information of a different nature relating to entrepreneurship was collected, which will not be discussed in this paper, and responses to a scale of obstacles to entrepreneurship which is the objective of this work. The scale of obstacles to entrepreneurship was made up of 16 aspects that can potentially be considered as barriers to entrepreneurship (López, 2009; Ruiz-Ruano, 2015). The potential obstacles posed were the following: having to work too many hours (o1), lack of theoretical training (o2), difficulties in obtaining capital (o3), lack of creativity to find original/innovative ideas (o4), huge responsibility (o5), lack of experience (o6), problems managing people (o7), excessive tax burdens (o8), high risk (o9), fear of failure (o10), irregular income (o11), negative image of the entrepreneur (o12) ), being currently employed (o13), family aspects (lack of family support...) (o14), ignorance of the activity sector (o15), and bureaucratic procedures to set up a company (o16). The students responded to each of these potential obstacles using a scale of one to five, which indicated the degree to which they believed

Revista de Educación, 386. October-December 2019, pp. 61-84 69 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

that each element was an obstacle in order to study whether the degree of perceived importance of each obstacle shared variability with some another variable of the study. Later this score was recoded to zeros and ones using three as a cut-off point (a response of three or less was coded with a zero and the higher scores were coded as one) with the objective of categorizing which potential obstacles were really problematic for the student body. For their part, the teaching staff responded by indicating if they considered the item represented an obstacle (coded with a one) or not (coded with a zero).

Procedure

To administer the questionnaires to the University of Almeria students, probabilistic sampling was carried out with proportional allocation according to degree courses. The minimum size initially estimated for the sample was 1,200 cases (which was satisfied) and the proportion of cases for each of the degree courses were also adjusted to the distribution of the university student population. We contacted the teaching staff in the selected degree courses and asked for permission to administer the questionnaire at the end or start of the class sessions. The data collection related to the university teaching staff was managed through an electronic form developed with LimeSurvey. A database of emails from all the available teaching staff was created (14,665 records) in the corresponding websites of the Andalusian public universities and each of them was invited to take part in the study. Two invitations to participate in the study were sent out, separated by 15 days. The data collection was terminated one month after the last mailing of invitations given that the response rate was 18.21%, which exceeded the minimum 15% estimated at the beginning of the investigation.

Data Analysis

Data analysis was carried out under the perspective of the general research model based on correlation and regression analysis (Cohen, Cohen, West, & Aiken, 2003). In addition, the Bayes Factor was used

in favour of the alternative hypothesis (FB10) as an inferential tool to

70 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university compare the likelihood of this hypothesis against the null hypothesis (correlation equal to zero). To assess the relevance of the Bayes Factors obtained, Jeffreys proposal (1948) was taken into account. Thus, Bayes Factors between 1 and 3 were interpreted as anecdotal evidence leading to the alternative hypothesis, between 3 and 10 as substantive evidence, between 10 and 30 strong, between 30 and 100 very strong, while those greater than 100 were considered decisive. The calculation of Bayes Factors was done using the free and open source JASP program (JASP Team, 2018). Previous distributions for all analyses were based on Cauchy distributions with r = 1 rescaling factor since recent simulation studies suggest that they contribute in a balanced way to statistical decision making (Jeon & De Boeck, 2017). To study the structural or qualitative relationship established between the potential obstacles to entrepreneurship, the bnlearn package for R version 4.2 was used (Scutari, 2010). Given that the objective was to estimate the more parsimonious structural models that describe the potential obstacles for each participant profile, the tabu search algorithm was used using the Bayesian Information Criterion (BIC) as an optimization statistic to prevent the algorithm from being blocked at local minimums (Scutari & Denis, 2014).

Results

As can be seen in Table I, the obstacles that are principally linked to entrepreneurial intention among university teaching staff and students are not the same. While the obstacles most related to the entrepreneurial intention in the university teaching staff are 4 (lack of creativity), 8 (tax burdens), 15 (ignorance of the sector) and 16 (bureaucratic procedures); those which are most linked to entrepreneurial intent among students are 1 (too many hours of work), 4 (lack of creativity), 7 (problems managing people) and 10 (fear of failure). It is also observed that the direction of these potential barriers is perceived differently by both groups. While all obstacles are negatively related to entrepreneurial intention in the case of students, the teaching staff considered the presence of bureaucratic procedures and tax burdens as obstacles which are positively related to entrepreneurial intention.

Revista de Educación, 386. October-December 2019, pp. 61-84 71 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

TABLE I. Correlation, p-value, Bayes Factor (FB10) and Bayesian credibility interval (95% CI) between each of the obstacles (o1-o16) and entrepreneurial intention.

Teaching Staff Students IC IC Variable r p FB r p FB 10 95% 10 95% o1 -.03 .263 0.06 -.08 .02 -.10 < .001 22.34 -.16 -.05 o2 -.08 .002 3.69 -.13 -.03 -.05 .076 0.17 -.11 .01 o3 .06 .021 0.45 .01 .11 -.02 .510 0.05 -.08 .04 o4 -.10 < .001 43.34 -.15 -.05 -.13 < .001 595.05 -.18 -.07 o5 -.05 .072 0.16 -.10 .004 -.08 .004 2.25 -.14 -.03 o6 -.03 .175 0.08 -.08 .02 -.08 .006 1.66 -.14 -.02 o7 -.05 .037 0.28 -.10 -.003 -.10 < .001 17.47 -.16 -.05 o8 .14 < .001 95013.70 .09 .19 -.02 .496 0.05 -.08 .04 o9 .05 .079 0.15 -.01 .09 -.09 .003 3.33 -.14 -.03 o10 .01 .655 0.04 -.04 .06 -.11 < .001 36.89 -.16 -.05 o11 .01 .652 0.04 -.04 .06 -.07 .015 0.69 -.13 -.01 o12 -.01 .803 0.03 -.06 .04 -.03 .257 0.07 -.09 .02 o13 -.03 .196 0.07 -.08 .02 -.05 .106 0.13 -.10 .01 o14 .02 .345 0.05 -.03 .07 -.04 .145 0.10 -.10 .01 o15 -.14 < .001 68636.71 -.19 -.09 -.09 .002 3.81 -.14 -.03 o16 .12 < .001 4399.55 .07 .17 -.05 .080 0.17 -.11 .01

Source: own production.

In Figure I we can see that ignorance of the activity sector (o15) and bureaucratic procedures (o16) have a direct impact on the entrepreneurial intention among university teaching staff. On the other hand, their perception of excessive tax burdens (o8), despite having the highest degree of relationship with entrepreneurial intention, is related to this through the problematic perception of bureaucratic procedures. On the other hand, the relationship observed between entrepreneurial intention and the lack of creativity (o4) appears to be mediated by a lack of knowledge about the activity (o15) among the university teaching staff. With regard to the entrepreneurial intention of university students, a lack of creativity (o4) is the potential obstacle that is directly related to the intention to create a company. In addition, we can observe that the impact of perceived problems managing people (o7) are mediated by that same lack of creativity.

72 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

There are three obstacles (lack of training, o2, difficulties in raising capital, o3, and problems managing people, o7) for teaching staff which, despite obtaining low p-values from a classical point of view, do not manage to obtain Bayes factors large enough to reinforce the idea that the relationship established with the entrepreneurial intention is, at least, strong. It should also be noted that the relationship observed between obstacle o3 and intention seems to condition the rest of the potential obstacles given its position in the graph as the mother variable of the rest of the structure.

FIGURE I. Bayesian network structures for the relationship between obstacles and entrepreneurial intention (int) in teaching staff (left) and students (right). The non-relevant nodes have been removed from the graphs.

Source: own production.

Table I also shows four potential obstacles for students (lack of experience, o6, high risk, o9, irregular income, o11, and ignorance of the sector, o15) that would be related to the entrepreneurial intention from a classical point of view attending to p-value. However, the Bayes Factors associated with these potential obstacles do not suggest that the recorded sample evidence can be considered to be strongly in favour of the relationship between each pair of variables. As can be seen in Figure I, the relationship between obstacle o15 and entrepreneurial intention could be explained by the mediation of lack of creativity (o4) while for

Revista de Educación, 386. October-December 2019, pp. 61-84 73 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

others it could be explained by its relationship with perceived problems to manage people (o7). As can be seen in Table II, the potential obstacles that most relate to the creation of companies among university teaching staff are the lack of creativity to find innovative ideas (o4), irregular income (o11) and ignorance of the sector (o15). However, the last two are those that directly affect the creation of companies among university teaching staff (Figure II). While irregular income is often considered a problem by lecturers who create companies, ignorance of the sector of activity is perceived as a problem by the faculty sector that has not created a company. In addition, difficulties to raise capital (o3), tax burdens (o8), the negative image of the entrepreneur (o12) and problems managing people (o7) also show a notable relationship with the creation of companies among the teaching staff.

TABLE II. Correlation, p-value, Bayes Factor (FB10) and Bayesian credibility interval (95% CI) between each of the obstacles (o1-o16) and the creation of companies.

Teaching Staff Students

Variable r p FB10 IC 95% r p FB10 IC 95% o1 .05 .013 0.55 .01 .09 -.11 < .001 50.11 -.16 -.05 o2 -.06 .003 1.87 -.10 -.02 -.09 .002 4.10 -.14 -.03 o3 -.08 < .001 26.24 -.11 -.04 -.03 .373 0.05 -.08 .03 o4 -.12 < .001 3.68 × 106 -.16 -.08 -.11 < .001 55.41 -.17 -.05 o5 .06 .003 1.94 .02 .10 -.08 .008 1.26 -.13 -.02 o6 -.05 .021 0.36 -.09 -.01 -.10 < .001 14.62 -.16 -.04 o7 .06 .002 3.27 .02 .10 -.02 .471 0.05 -.08 .04 o8 .07 < .001 19.87 .03 .11 -.01 .855 0.04 -.06 .05 o9 -.06 .006 1.03 -.10 -.02 -.03 .354 0.06 -.08 .03 o10 -.05 .012 0.58 -.09 -.01 -.10 < .001 12.76 -.15 -.04 o11 .14 < .001 1.44 × 108 .10 .17 -.05 .102 0.14 -.10 .01 o12 .07 < .001 9.42 .03 .11 -.03 .337 0.06 -.08 .03 o13 -.03 .166 0.07 -.07 .01 .02 .430 0.05 -.03 .08 o14 .02 .389 0.04 -.02 .06 -.02 .550 0.04 -.07 .04 o15 -.16 < .001 1.58 × 1012 -.20 -.12 -.11 < .001 39.28 -.16 -.05 o16 .09 < .001 511.94 .05 .13 -.06 .043 0.28 -.11 -.002

Source: own production.

74 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

As can be seen in Table II, there are six potential obstacles that are related to the creation of companies for university students. All these obstacles are associated negatively with the fact of creating a company which means that those students who have not created a company consider these situations as obstacles to entrepreneurship. The three obstacles that obtain a Factor of Bayes higher than 30 are the fact of having to work too many hours (o1), a lack of creativity to find original or innovative ideas (o4) and ignorance of the activity sector (o15). However, as Figure II shows, having to work too many hours seems to condition the creation of companies among university students. University students who have not created a company also perceive a lack of experience (o6) and fear of failure (o10) as problems when setting up their own business, although in both cases the associated Bayes Factor is between 10 and 30. Finally, the Bayes Factor linked to the lack of theoretical training (o2) is also greater than three and, therefore, could be considered as an element that conditions the creation of companies among university students. Figure II shows that having to work too many hours (o1) is the obstacle that directly affects the creation of companies among students. It is also appreciated that this obstacle leads to irregular income (o11) and this, in turn, leads to a fear of failure (o10).

FIGURE II. Bayesian network structures for the relationship between obstacles and creation of companies (crea) in teachers (left) and students (right). The non-relevant nodes have been removed from the graphs.

Source: own production.

Revista de Educación, 386. October-December 2019, pp. 61-84 75 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Conclusions

As has been observed, there is a relationship between the perception of obstacles, entrepreneurial intention and the creation of companies among both teachers and students. It is worth noting, however, that the direction of the relationship between these variables is not the same for teachers as for students. In the case of students, the relationship between the perception of obstacles, entrepreneurial intention and the creation of companies is negative. That is to say, the higher the perception of obstacles, the lower the intention and the creation of companies. On the contrary, in the case of teaching staff, the relationship is not always negative, with positive relationships being observed in certain cases. This could be interpreted alluding to the fact that the teaching staff intend to create companies even knowing that they will encounter certain types of obstacles while students will not. The impact of the economic component on the entrepreneurial intention of teaching staff, as previous studies suggest (e.g., Iakovleva, 2014; Nabi et al., 2006; Rodeiro et al., 2012), is also noticeable. As can be seen in Figure I, it seems that the availability of economic resources conditions the perception of the remaining obstacles to entrepreneurship for the teaching staff. There is only one common obstacle perceived by both the teaching staff and students, and which appears to be related both to entrepreneurial intention and to the creation of companies. It is obstacle 4, lack of creativity to find original/innovative ideas. This result is also in line with the work of Iakovleva et al. (2014), in which they reported that the lack of certain skills and/or abilities conditioned the creation of new companies. Our results also suggest that obstacles related to irregular income, fear of failure and working too many hours have a notable role in the creation of companies in the university context (Borbera et al., 2014; Deh et al., 2013). The estimated networks are also consistent with the idea that there is a set of obstacles relating to bureaucracy when we approach the creation of companies among university students and teaching staff (Deh et al., 2013). Knowing which obstacles most relate to the entrepreneurial intention and creation of companies, actions could be designed to guide the potential entrepreneur to achieve their objectives. As we have pointed out throughout the paper, there are still few actions developed along these lines at university. As pointed out by Santos (2013) and Santos and

76 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Muñoz-Rodríguez (2015, 2017), the Spanish labour market today does not have the capacity to employ all of its graduates, so that other actions, such as international mobility, must inevitably be considered in order to be able to be active in the labour market or, as entrepreneurs, in their own company. That is why, encouraging entrepreneurial spirit among students is positive so that university graduates will be able to obtain greater benefits from academic training, as pointed out by Toledano (2006). Accordingly, the university should take advantage of the entrepreneurial experience of teaching staff as a way to bring together the university and the company. If this experience was little or null, taking an idea from Toledano (2006) and Rodeiro et al. (2012), it would be necessary to give them training in skills and an entrepreneurial culture, an aspect historically requested by students to try to bring the university closer to the real world. If the desire is genuine to make an entrepreneurial university a reality with a social and economic contribution, doing so will be difficult to achieve starting from the traditional conception of education, as suggested by Marina (2010). In this sense, we believe that in Spain it is still necessary to change many aspects of the university context in general to help improve the employability prospects of graduates. With regard to the perception of obstacles, it is important that students and potential entrepreneurs have a more realistic perception of the processes. This can be obtained through the stories of real entrepreneurs. Programs for the promotion of entrepreneurial intention could also be implemented within the curricula of undergraduate students. These programs could train students in five major facets (Nabi et al., 2016): knowing why (know-why), what (know-what), how (know-how), who (know-who), and the when (know-when) to create a company. With regard to the limitations of our work, we wish to highlight the representativeness of the sample, since it has been limited to the Andalusian region (for teaching staff) and the University of Almeria (for students). Future studies should try to make comparisons between regions to see if these results are maintained. On the other hand, it may be that current entrepreneurs and potential entrepreneurs taken into consideration different obstacles to those that contemplated in this study, and that may have a greater or lesser impact on the entrepreneurial intention. In addition, this paper does not explore whether the implementation of strategies to promote entrepreneurship in the university entails a reduction in the perception of long-term barriers. In

Revista de Educación, 386. October-December 2019, pp. 61-84 77 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

this sense, it would be interesting for future studies to track over time those potential entrepreneurs who receive advice from the university. At the same time, it would help again to have a measure of the effectiveness of improvement plans for the enterprise. Finally, we believe that there are certain variables that have not been taken into account and that could condition the results obtained and future work should analyse these. For example, the socio-economic context of the participants as well as their gender have not been considered.

Bibliographical References

Abbott, A., Zastrow, M., Gibney, E., & Nordling, L. (2014, 16 de octubre). The university experiment: campus as laboratory. Nature, 514, 288- 291. doi: 10.1038/514288a Altman, N. & Krzywinski, M. (2018). The course(s) of dimensionality. Nature Methods, 15, 339-400. doi: 10.1038/s41592-018-0019-x Ajzen, I., & Fishbein, M. (2005). The influence of attitudes on behavior. En D. Albarracín, B. T. Hohnson, & M. P. Zanna (Eds.), The hadnbook of attitudes (pp. 173–221). Mahwah, NJ: Erlbaum. Anscombe, F. J. (1961). Bayesian statistics. The American Statistician, 15, 21-24. doi: 10.2307/2682504 Armitage, C. J., & Conner, M. (2001). Efficacy of the Theory of Planned Behaviour: a meta-analytic review. British Journal of Social Psychology, 40, 471-499. Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437. Barba-Sánchez, V., & Atienza-Sahuquillo, C. (2018). Entrepreneurial intention among engineering students: The role of entrepreneurship education. European Research on Management and Business Economics, 24, 53-61. doi: 10.1016/j.iedeen.2017.04.001 Bernal, A. (2014). Competencia emprendedora e identidad personal. Una investigación exploratoria con estudiantes de Educación Secundaria Obligatoria. Revista de Educación, 363, 384-411. doi: 10.4438/1988- 592X-RE-2012-636-192

78 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Block, J. H., Hoogerheide, L., & Thurik, R. (2013). Education and entrepreneurial choice: An instrumental variables analysis. International Small Business Journal: Researching Entrepreneurship, 31(1), 23-33. doi: 10.1177/0266242611400470 Bolstad, W. M. (2007). Introduction to Bayesian statistics (2nd Ed.). Hoboken, NJ: Wiley. Borbera, D., Lekovic, B., & Berber, N. (2014). Comparative analysis of entrepreneurship barriers: findings from Serbia and Montenegro. Inzinerine Ekonomika–Engineering Economics, 25(2), 167-176. doi: 10.5755/j.01.ee.25.2.3113 Cohen, J. (1994). The Earth is round (p < .05). American Psychologist, 49, 997-1003. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioural sciences (3rd Ed.). New York: Routledge. Cowell, R. G., Dawid, A. P., Lauritzen, S. L., & Spiegelhalter, D. J. (1999). Probabilistic networks and expert systems. Harrisonburg, VA: Springer. Crum, M., & Chen, Y. (2015). Self-employment and subjective well-being: A multi-country analysis. International Journal of Entrepreneurship, 19, 53-61. doi: 10.1016/j.iedeen.2017.04.001 Deh, I. Y., Yeboah, S., & Agyemang, I. K. (2013). The link between demographics and perceive barriers to entrepreneurship. Journal of Small Business and Entrepreneurship Development, 1, 42-50. Dohse, D., & Walter, S. G. (2012). Knowledge context and entrepreneurial intentions among students. Small Business Economics, 39, 877-895. doi: 10.1007/s11187-011-9324-9 Editorial Nature (2014, 16 de Octubre). Universities challenged. Nature, 514, 273. doi: 10.1038/514273a Edwards, W., & Fasolo, B. (2001). Decision technology. Annual Review of Psychology, 52, 581–606. Etzkowitz, H., Webster, A., Gebhardt, C., & Cantisano, B. R. (2000). The future of the university and the university of the future: evolution of ivory tower to entrepreneurial paradigm. Research Policy, 29, 313-330. Glassman, A. M., Moore, R. W., Rossy, G. L., Neupert, N. K., Jones, D. E., & Harvey, M. (2003). Academic entrepreneurship. Views on balancing the Acropolis and the Agora. Journal of Management Inquiry, 12(4), 353-374. doi: 10.1177/1056492603258979

Revista de Educación, 386. October-December 2019, pp. 61-84 79 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Haller, M. K., & Welch, E. W. (2014). Entrepreneurial behavior of academic scientists: network and cognitive determinants of commitment to grant submissions and award outcomes. Entrepreneurhip Theory and Practice, 38, 807-831. doi: 10.1111/etap.12022 Halsey, L. G., Currant-Everett, D., Vowler, S. L., & Drummond, G. B. (2015). The fickled P value generates irreproducible results. Nature Methods, 12, 179-185. doi: 10.1038/nmeth.3288 Henry, C., Hill, F., & Leitch, C. (2005). Entrepreneurship education and training: can entrepreneurship be taught? Part I. Education + Training, 47(2), 98-111. doi: 10.1108/00400910510586524 Iakovleva, T. A., Kolvereid, L., Gorgievsky, M. J., & Sørhang, Ø. (2014). Comparison of perceived barriers to entrepreneurship in eastern and western european countries. International Journal of Entrepreneurship and Innovation Management, 18, 115-133. doi: 10.1504/IJEIM.2014.062874 JASP Team (2018). JASP (Version 0.8.6) [Computer software]. Jefatura del Estado. (2001). Ley Orgánica 6/2001, de 21 de diciembre, de Universidades. BOE núm. 307. Madrid, España. Recuperado el 1 abril 2018 de: https://www.boe.es/buscar/pdf/2001/BOE-A-2001-24515- consolidado.pdf Jefatura del Estado. (2005). Real Decreto 415/2015, de 29 de mayo, por el que se modifica el Real Decreto 1312/2007, de 5 de octubre, por el que se establece la acreditación nacional para el acceso a los cuerpos docentes universitario. BOE núm. 144. Madrid, España. Recuperado el 1 abril 2018 de: https://www.boe.es/boe/dias/2015/06/17/pdfs/ BOE-A-2015-6705.pdf Jefatura del Estado. (2006). Ley Orgánica 2/2006, de 3 de mayo, de Educación. BOE núm. 106. Madrid, España. Recuperado el 1 de abril de 2018 de: https://www.boe.es/boe/dias/2006/05/04/pdfs/A17158- 17207.pdf Jefatura del Estado. (2013). Ley Orgánica 8/2013, de 9 de diciembre, para la mejora de la calidad educativa. BOE núm. 295. Madrid, España. Recuperado el 1 abril 2018 de: https://www.boe.es/buscar/pdf/2013/ BOE-A-2013-12886-consolidado.pdf Jeffreys, H. (1948). Theory of probability (2nd Ed.). Oxford: Oxford University Press. Jeon, M., & De Boeck, P. (2017). Decision qualities of Bayes Factor and p value-based hypothesis testing. Psychological Methods, 22, 340-360. doi: 10.1037/met0000140

80 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Jones, O., Macpherson, A., & Woollard, D. (2008). Entrepreneurial ventures in higher education. International Small Business Journal, 26, 683-708. doi: 10.1177/0266242608096089 Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773-795. doi: 10.1080/01621459.1995.10476572 Kuratko, D. F. (2005). The emergence of entrepreneurship education: Development, trends, and challenges. Entrepreneurship Theory and Practice, 29, 577-598. doi: 10.111/j.1540-6520.2005.00099.x Lüthje, C., & Franke, N. (2003). The ‘making’ of an entrepreneur: testing a model of entrepreneurial intent amogn engineering students at MIT. Research and Development Management, 33(2), 135-147. López, J. (2009). Modelos predictivos en actitudes emprendedoras: análisis comparativo de las condiciones de ejecución de las redes bayesianas y la regresión logística. Tesis doctoral, Facultad de Psicología, Universidad de Almería. URI: http://hdl.handle.net/10835/356 Marina, J. A. (2010). La competencia de emprender. Revista de Educación, 351, 49-71. Mars, M. M. (2009). College student eco-entrepreneurship: a social movement perspective. Advances in the Study of Entrepreneurship, Innovation and Economic Growth, 20, 153-172. doi: 10.1108/S1048- 4736(2009)0000020010 Ministerio de Industria, Economía y Empresa (2014). José Manuel Soria y José Ignacio Wert presentan la segunda edición del programa Iniciativa Emprendedora en la universidad. [online] Disponible en: http://www.minetur.gob.es [Recuperado el 3 de Abril de 2018]. Ministerio de Industria, Economía y Empresa (2015). Los ministros de Industria, Energía y Turismo y de Educación, Cultura y Deportes presentan la tercera edición del programa Iniciativa Emprendedora en la universidad. [online] Disponible en: http://www.minetur.gob.es [Recuperado el 3 de Abril de 2018]. Nabi, G., Walmsley, A., Liñán, F., Akhtar, I., & Neame, C. (2016). Does entrepreneurship education in the first year of higher education develop entrepreneurial intentions? The role of learning and inspiration. Studies in Higher Education, 41, 452-467. doi: 10.1080/03075079.2016.1177716 Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences. doi: 10.1073/pnas.1708274114

Revista de Educación, 386. October-December 2019, pp. 61-84 81 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Oosterbeek, H., van Praag, M., & Jsselstein, A. (2010). The impact of entrepreneurship education on entrepreneurship skills and motivation. European Economic Review, 54, 442-454. doi: 10.1016/j. euroecorev.2009.08.002 Orlitzky, M. (2012). How can significance tests be deinstitutionalized? Organizational Research Methods, 5, 199-228. Patzelt, H., & Shepherd, D. A. (2009). Strategic entrepreneurship at universities: academic entrepreneurs’ assessment of policy programs. Entrepreneurship Theory and Practice, 33, 319-340. doi: 10.1111/j.1540-6520.2008.00291.x Plaschka, G. R., & Welsch, H. P. (1990). Emerging structures in entrepreneurship education: Curricular designs and strategies. Entrepreneurship Theory and Practice, 14(3), 55-71. Puga, J. L., Krzywinski, M., & Altman, N. (2015). Points of Significance: Bayesian networks. Nature Methods, 12, 799-800. doi: 10.1038/ nmeth.3550 Robinson, P. B., Stimpson, D. V., Huefner, J. C., & Hunt, H. C. (1991). An attitude approach to the prediction of entrepreneurship. Entrepreneurship Theory and Practice, 15, 13-31. Rueda, I., Fernández-Laviada, A., & Herrero, A. (2014). Intención de emprendedimiento: ventajas e inconvenientes percibidos. Academia. Revista Latinoamericana de Administración, 27, 284-315. doi: 10.1108/ARLA-09-2013-0144 Rueda, I., Sánchez, L., Herrero, A., Blanco, B., & Fernández-Laviada, A. (2013). ¿Existen niveles adecuados de formación y financiación que incentiven la intención emprendedora? Revista FIR, FAEDPYME International Review, 2(3), 28-38. Rodeiro, D., Fernández, S., Rodríguez, A., & Otero, L. (2012). La financiación de la investigación como motor del emprendimiento académico: análisis de las patentes universitarias. Revista de Educación, 357, 203- 229. doi: 10.4438/1988-592X-RE-2011-357-058 Ruiz-Ruano, A. M. (2015). Aprendizaje estructural de redes bayesianas para modelar el emprendimiento académico de base sostenible y tecnológica. Tesis doctoral, Facultad de Ciencias de la Salud, UCAM Universidad Católica de Murcia. URI: http://hdl.handle.net/10952/1556 Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284.

82 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Sánchez, J. C. (2011). Entrepreneurship as a legitimate field of knowledge. Psicothema, 23, 427-432. Santos, A. (2013). Fuga de cerebros y crisis en España: los jóvenes en el punto de mira de los discursos empresariales. AREAS Revista Internacional de Ciencias Sociales, 32, 125-137. Santos, A., & Muñoz-Rodríguez, D. (2015). Fuga de cerebros y biografías low cost: nueva etapa en la precarización de la juventud. Recerca, Revista de Pensament i Anàlisi, 16, 13-33. doi: 10.6035/Recerca.2015.16.2 Santos, A., & Muñoz-Rodríguez, D. (2017). La subjetivación del capital humano: la movilidad como inversión en los discursos de los jóvenes universitarios que han protagonizado la fuga de cerebros. AREAS Revista Internacional de Ciencias Sociales, 36, 127-139. Sardeshmukh, S. R., & Smith-Nelson, R. M. (2011). Educating for an Entrepreneurial Career: Developing Opportunity- Recognition Ability. Australian Journal of Career Development, 20(3), 47-55. doi: 10.1177/103841621102000308 Scutari, M. (2010). Learning Bayesian Networks with the bnlearn R package. Journal of Statistical Software, 35(3), 1-22. doi: 10.18637/ jss.v035.i03 Scutari, M., & Denis, J. B. (2014). Bayesian networks: with examples in R. Boca Raton, FL: CRC Press. Shapero, A. (1985). Why entrepreneurship? A worldwide perspective. Journal of Small Business Management, 23(4), 1-5. Souitaris, V., Zerbinati, S., & Al-Laham, A. (2007). Do entrepreneurship programmes raise entrepreneurial intention of science and engineering students? The effect of learning, inspiration and resources. Journal of Business Venturing, 22, 566-591. doi: 10.1016/j.jbusvent.2006.05.002 Stern, H. S. (2016). A test by any other name: P-values, Bayes Factors and statistical inference. Multivariate Behaviour Research, 51, 23-39. doi: 10.1080/00273171.2015.1099032 Toledano, N. (2006). Las perspectivas empresariales de los estudiantes universitarios: un estudio empírico. Revista de Educación, 341, 803- 825. Van Praag, C. M., & Versloot, P. (2007). What is the value of entrepreneurship? A review of recent research. Small Business Economy, 29, 351-382. doi: 10.1007/s11187-007-9074-x Van Zyl, C. J. J. (2018). Frequentist and bayesian inference: a conceptual premier. New Ideas in Psychology, 51, 44-49. doi: 10.1016/j. newideapsych.2018.06.004

Revista de Educación, 386. October-December 2019, pp. 61-84 83 Received: 26-07-2018 Accepted: 21-06-2019 Ruíz-Ruano García, A. M., María del Pilar Casado Belmonte, M. P., López Puga, J. Bayesian analysis of barriers towards entrepreneurship at university

Von Graevenitz, G., Harhoff, D., & Weber, R. (2010). The effects of entrepreneurship education. Jorunal of Economic Behavior and Organization, 76(1), 90-112. doi: 10.1016/j.jebo.2010.02.015 Wasserstein, R. L., & Lazar, N. A. (2016). The ASA’s statement on p-values: context, process, and purpose. The American Statistician, 70, 129- 133. doi: 10.1080/00031305.2016.1154108 Watson-Capps, J. J., & Cech, T. R. (2014, 16 de Octubre). Academia and industry: companies on campus. Nature, 514, 297-298. doi: 10.1038/514297a

Contact address: Ana María Ruiz-Ruano García. UCAM Universidad Católica de Murcia, Facultad de Ciencias Jurídicas y de la Empresa, Departamento de Ciencias Sociales, Jurídicas y de la Empresa. Campus de los Jerónimos s/n, 30107, Guadalupe, Murcia. E-mail: [email protected]

84 Revista de Educación, 386. October-December 2019, pp. 61-84 Received: 26-07-2018 Accepted: 21-06-2019 Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation1

Las Rúbricas No neutralizan el Efecto de los correctores: Una estimación con el modelo de facetas múltiples de Rasch

DOI: 10.4438/1988-592X-RE-2019-386-428

Rubén Fernández-Alonso Pamela Woitschach José Muñiz Universidad de Oviedo

Abstract Constructed response items are widely used in all types of educational evaluations. Despite the use of very specific rubrics for scoring items, the influence of raters is well documented in the literature, affecting the results of evaluations. The main goal of the present study is to estimate the effects of raters and rubrics in the assessment of written expression. We used 13 raters who reviewed 375 pieces of writing by sixth grade students. The raters were assigned to the written essays following a 13-block Youden square, a balanced incomplete block design. In the data analysis, the many-faceted Rasch model was used with three facets: corrector, rubrics and difficulty of passing. Different models were compared, and the effects of the raters and the characteristics of the rubrics were analyzed. The results show the differences between raters in terms of severity and accuracy of evaluation. We conclude that not considering the effect of the rater in the estimation of student outcomes can introduce a clear component of inequity in the evaluations.

Key words: Educational assessment, rater, scoring rubrics, MRFM models

(1) This research was funded by the Ministry of Economy and Competitiviness of the Spanish Government. References: PSI2017-85724-P, BES2012-053488.

Revista de Educación, 386. October-December 2019, pp. 85-107 85 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Resumen Los ítems de respuesta construida son ampliamente utilizados en todo tipo de evaluaciones educativas. A pesar de la utilización de rúbricas muy específicas para su corrección, la influencia de los correctores está bien documentada en la literatura, afectando a los resultados de la evaluación. El objetivo central del presente trabajo es la estimación de los efectos de los correctores y de las rúbricas en una tarea de expresión escrita. Se utilizaron 13 correctores que revisaron 375 producciones escritas de estudiantes de sexto curso. Los correctores fueron asignados a los ensayos escritos siguiendo un cuadrado Youden de 13 bloques, un diseño de bloques incompletos balanceado. En el análisis de datos se empleó el modelo de Rasch de facetas múltiples con tres facetas: corrector, rúbricas y dificultad de paso de los mismos. Se compararon diferentes modelos yse analizaron los efectos del corrector y las características de las rúbricas. Los resultados ponen de manifiesto las diferencias entre los correctores en cuanto la severidad y la exactitud de sus juicios. Se concluye que no incluir el efecto del corrector en la estimación de los resultados del alumnado puede introducir un componente claro de inequidad en las evaluaciones.

Palabras clave: Evaluación educativa, correctores, rúbricas, Modelos MFRM

Introduction

European education systems include high-stakes exams, such as exams for qualifications, certifications, access to further study and school evaluations. They include items where rater effects can influence a student’s score (European Commission/EACEA/Eurydice, 2009). Given how important these evaluations are to the student, society, and the various stakeholders involved, their objectivity has been widely researched (Congdon & McQueen, 2000; Engelhard, 1992; Gyagenda & Engelhard, 2009; Leckie & Baird, 2011; Lunz & Stahl, 1990; Lunz, Wright & Linacre, 1990; Park, 2010; Wolfe, 2004). According to Eckes (2009) there are two broad sources of error that affect the objectivity of evaluations: distal and proximal. Distal factors refer to aspects such as the characteristics of the evaluator, the student, and the context of the evaluation. Evaluator variation has been found associated with gender, age, professional history, the field the rater works in, as well as in raters’ attitudes towards the gender,

86 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation nationality or ethnicity of the person being evaluated (Amengual, 2004; Congdon & McQueen, 2000; Linacre, Engelhard, Tatum & Myford, 1994; Lopes Toffoli, de Andrade & Bornia, 2015; Mahmood, Dagnæs, Bube, Rohrsted & Konge, 2017). Differences linked to the grading process have also been found, such as previous experience, the grading order, entertainment, familiarity with rubrics, and rater cognitive strategies (Bejar, 2012; Congdon & McQueen, 2000; Crisp, 2012; Jonsson & Svingby, 2007; Leckie & Baird, 2011; Linacre et al., 1994; McNamara, 1996; Suto, 2012). Proximal factors refer to the construct being measured, the difficulty of the task and the grading criteria, the structure of the scoring banks and rater effects (Eckes, 2009). Cuxart-Jardí (2000), Cuxart-Jardí, Martí- Recober and Ferrer-Juliá (1997) found better agreement in scientific and mathematics subjects than in socio-linguistic subjects. Jonsson and Svingby (2007) reported differences in reliability according to the number of categories used and the nature of the task, concluding that indexes of agreement increase as the number of categories decreases, and that physical tests and case studies exhibit higher reliability indexes than oral or written production. Finally, Baird, Meadows, Leckie and Caro (2017), Jonsson and Svingby (2007) and Kuo (2007) agree when they indicate that in high-stakes tests analytical rubrics are preferable to holistic rubrics, although the procedures, scoring and criteria for discerning between performance levels need to be clearly detailed with examples. The proximal factors that have received most attention include rater effects (strictness/leniency, halo, range restriction, consistency, etc.), which have been studied from various methodological perspectives such as the classical approach (OECD, 2014; Saal, Downey & Lahey, 1980; Woitschach et al., 2018), generalizability theory (Sudweeks, Reeve & Bradshaw, 2005), multilevel analysis (Congdon & McQueen, 2000; Leckie & Baird, 2011), and Item Response Theory models (TRI; Adams & Wu, 2010; Lunz et al., 1990; Myford & Wolfe, 2003, 2004). While each strategy has its strengths and weaknesses (Baird, Hayes, Johnson, Johnson & Lamprianou, 2013; Gyagenda&Engelhard, 2009; Sudweeks et al, 2005), TRI models offer integrated solutions, as they allow rubrics to be validated (Lallmamode, Mat Daud & Abu Kassim, 2016), they examine the effects of various sources of variation (Linacre et al., 1994), and they manage those effects as a whole in order to estimate student results (Eckes, 2009).

Revista de Educación, 386. October-December 2019, pp. 85-107 87 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

The Many-Facet Rasch Measurement (MRFR; Linacre et al., 1994) allows the parameterization in one scale of all of the facets of the model; subject competence, rater characteristics, difficulty, category parameters, and how many facets the analysis could include (Eckes, 2009; Myford & Wolfe, 2003, 2004; Prieto-Adánez, 2011). In addition, Linacre et al. (1994) and Lunz et al. (1990) demonstrated that MFRM could identify rater effects (severity/leniency, inconsistent scoring, over-consistency, etc.) and diagnose issues with grading rubrics and criteria (fit, scale dimensionality, category parameter, etc.). The main practical advantage of MFRM is that it has demonstrated the need to include model parameters (in particular variations in rater severity) in the estimation of student test results, as otherwise this could affect decisions about levels of student competence, and ultimately the validity of study conclusions (Eckes, 2005, 2009; Prieto-Adánez, 2015; Shackleton, 2018). Within this context, the objective of this study is to examine rater effects in the evaluation of writing, and its interaction with the rubrics used. This general objective can be separated into four specific goals: (a) Choose the multi-faceted Rasch model that best fits the data; (b) Examine the fit of the raters to the model and analyze the effects introduced by the raters; (c) Examine the effects of the rubrics and their fit to the model; and (d) Estimate the extent of bias added by the raters to the students’ grades. These objectives are particularly interesting as there are very few studies that have looked at the consequences for students of their work being assigned to raters with differing levels of severity. In Spain, research up to now has focused on examining rater effects and the fit of models, without estimating wrongly graded cases as they have not included the effects of the rater in determining student competence. We believe that our research is breaking new ground in this sense, in addition to providing novel methodologies in the design and analysis of assigning raters.

Method

Sample

We used 13 raters who were assigned texts following a 13-block Youden square design. This is an incomplete balanced Latin square which uses the first four replicas of the 13x13 Latin square (Ato & Vallejo, 2007;

88 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Cochran & Cox, 1974; Frey, Hartig & Rupp, 2009; Organization for Economic Co-operation and Development [OECD], 2014). To do this the writings from 18 classes were grouped in 13 grading groups (blocks in experimental design terms). Table I shows the 13 groups and the number of writings assigned to each. On average, each group had 28 writings, although groups TB09 to TB13 graded more as they had writings from two classes each.

TABLE I. Makeup of the 13 grading groups: classes assigned, number of writings to grade, and raters in each grading group

Grading groups Class(es) Nº of (blocks of assigned to the writings Raters in each grading group writings) group to grade TB01 Aula 01 28 C01 C09 C12 C10 TB02 Aula 02 26 C02 C01 C08 C11 TB03 Aula 03 25 C03 C02 C09 C05 TB04 Aula 04 25 C04 C03 C01 C06 TB05 Aula 05 24 C05 C11 C04 C12 TB06 Aula 06 23 C06 C05 C10 C08 TB07 Aula 07 23 C07 C13 C05 C01 TB08 Aula 08 22 C08 C12 C13 C03 TB09 Aula 09 & Aula 18 34 C09 C08 C07 C04 TB10 Aula 10 & Aula 17 36 C10 C04 C02 C13 TB11 Aula 11 & Aula 16 35 C11 C10 C03 C07 TB12 Aula 12 & Aula 15 36 C12 C07 C06 C02 TB13 Aula 13 & Aula 14 37 C13 C06 C11 C09

Note: Authors’ data

The final four columns in the table identify the four raters assigned to each group and allow us to check the consistency, balance and efficiency of the Youden design. The consistency of the design means that each rater is assigned four groups and that each writing is graded by four raters. For example, Grader 1 (C1) is in groups TB01, TB02, TB04 and

Revista de Educación, 386. October-December 2019, pp. 85-107 89 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

TB07. Similarly, the writings assigned to TB01 will be graded by C1, C9, C12 and C10. The design is completely balanced as each rater will grade the same set of writings with every other rater on one occasion. For example, C1 will grade with C09, C12 and C10 in group TB01, with C02, C11 and C08 in group TB02, with C03, C04 and C06 in group TB04 and the remaining three raters in group TB07. The remaining groupings can be seen in table 1. The efficiency of the design is that each rater will, on average, grade a little over 100 writings (for example rater C12 will grade the 110 writings in sets TB01, TB05, TB08 and TB12) rather than the 375 that a full Latin square would need. In total 1500 writings will be graded in this study rather than the almost 5000 that the full design would need, saving about 60% of possible gradings without the conclusions of the analysis losing validity. In short, this distribution of raters and writings ensures the dual control of the Latin square (Fernández-Alonso & Muñiz, 2011) such that the possible severity or leniency of the raters cannot be imputed to being assigned a skilled or less-skilled writing group. The writings in this study come from the final evaluation of primary education (the Evaluación Final de Educación Primaria; required by organic law 8/2013 on Education Quality Improvement) carried out in a Spanish Autonomous community in academic year 2015/16, which collected 6653 writings from 403 class groups of sixth grade primary education. Using systematic, random sampling, 18 classes were selected with a probability proportional to their size (OECD, 2014). All of the students’ writings from each chosen class were graded, 375 in total.

Instruments

The prompt used in the evaluation was a text containing tourist information for three cities. The students had to choose one city and write a text to convince their classmates that it would be the best destination for a class trip. The grading rubric evaluated three constructs or cognitive processes in the official Spanish curriculum. Table II shows how the standards and processes are organized. The scores in the standards are summarized in the three constructs which were coded in a three-level scale as follows; 0 points = not achieved; 1 = achieved; and 2 = Consolidated. The test, specifications and rubrics are available from the Ministry of Education,

90 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Culture and Sport (2016, pp. 97-117) and may be found at: https://sede. educacion.gob.es/publiventa/descarga.action?f_codigo_agc=18314

TABLE II. Constructs or cognitive processes evaluated and standards in the grading rubric

Constructs or cognitive processes Rubric standards Organize ideas clearly and with thematic Coherence progression; express opinions, thoughts and evaluations coherently. Use connectors and punctuation to produce Cohesion a cohesive text; use pronoun substitution and synonyms to avoid repetition. Use appropriate grammar and spelling; use an appropriate register for the reader and topic; Appropriateness and presentation present a tidy writing that is clear, precise and ordered.

Procedure

Permission was first obtained so that the relevant education department could partly release the files containing the tests from the selected classes, in accordance with Spanish data protection laws (Organic law 15/1999 on protection of personal data). Each piece of writing, originally in pencil on paper, was scanned, anonymized via an identification code, edited with the Adobe Acrobat Prof DC® program to remove original grading marks, and saved in a file labelled with the identification code. In addition, 13 grading templates were prepared (one for each rater) containing the data about the writings assigned to each rater. An online platform was designed where all 375 writings were uploaded in which each rater only had access to their assigned files. The raters recorded the results of their grading on the online platform, and once the grades were uploaded, the codes were verified to avoid transcription errors. The raters, postgraduate Education students, received six hours of training in grading procedures, two hours of training in basic concepts of grading via rubrics, and four hours of work with the test instruments, including training in grading similar writings to those used in the study to familiarize themselves with the material and the grading process.

Revista de Educación, 386. October-December 2019, pp. 85-107 91 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Data analysis

We used the MFRM for polytomous data, which assumes that each student’s score is a logistical function which includes three facets in addition to student competence: grader (rater), cognitive processes (criteria) and difficulty of those processes step( ). This lets us define a model of principal effects and additional models of interaction between facets, and lets us perform a hierarchical comparison of models. Initially, the model of principal effects is created, then three models of interaction of two facets (models of second level interaction: criteria x step; rater x step; and criteria x rater), and finally a model which includes the interaction between the three facets together (criteria x step x rater). To see which model demonstrated the best fit, we used the χ2 test, comparing the reduction in deviance in each new model with the increase in the number of parameters necessary to produce models of growing complexity (Adams & Wu, 2010). Once the model was selected, and to meet the second study objective, the suitability of the scores was checked via the values of the raters’ mean square residuals (MNSQ), following the criteria from Eckes (2005) and Wolfe and McVay (2012). Rater effects (difficulty, halo and central tendency) were examined via comparison of the estimators of rater severity, analysis of separation indices between raters and the MNSQ fit values for the raters according to the criteria from Eckes (2005, 2009), Lunz et al., (1990), Engelhard (1994) and Wolfe and McVay (2012). The dimensionality of the scale, based on the supposed positive correlation between the scores of the three cognitive processes being evaluated, was verified by considering the MNSQ values of fit ofthe cognitive processes to the proposed model. Similarly, the values of position and indices of separation between the cognitive processes were analyzed. Finally, to meet the fourth study objective, we compared the classical scores of the rater groups (which only consider the rater grades) with estimations of student competency offered by the MFRM models, in which student scores are corrected for the effect of rater severity and the difficulty of the cognitive processes being evaluated. All analysis was performed using ConQuest 2.0 software (Wu, Adams, Wilson & Haldane, 2007).

92 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Results

Model fit

Table III gives the values of deviance and the number of parameters in each model. Model four exhibits the best fit to the data, modelling the differences between raters in the interpretation of the rubrics of the cognitive processes which, as will be seen later, is one of the main problems of the rubric used.

TABLE III. Comparison of models: deviance values and number of parameters

Deviance Nº of parameters

Model 1: Principal effects 7376.3 17

Model 2: Interaction criteria x step 7339.1 20

Model 3: Interaction rater x step 7268.0 30

Model 4: Interaction rater x criteria 6800.6 41

Model 5: Interaction rater x criteria x step 7045.9 56

Note: Authors’ data

Rater effects

Table IV shows the levels of each rater’s severity and the standard error of this estimation, the values of fit (MNSQ Uw-fit and MNSQ W-fit) and their confidence intervals, and the result of the T-test.

Revista de Educación, 386. October-December 2019, pp. 85-107 93 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

TABLE IV. Rater effects

Unweighted fit Weighted fit Rater SE of Severity (Uw-fit) (W-fit) ID severity MNSQ (CI) T MNSQ (CI) T C01 -0.385 0.120 1.43 (0.73 - 1.27) 2.8 1.52 (0.71 - 1.29) 3.1 C02 0.758 0.100 1.19 (0.75 - 1.25) 1.5 1.21 (0.76 - 1.24) 1.6 C03 0.114 0.134 0.92 (0.73 - 1.27) -0.5 0.93 (0.74 - 1.26) -0.5 C04 0.634 0.107 0.92 (0.75 - 1.25) -0.6 0.90 (0.75 - 1.25) -0.8 C05 -1.050 0.132 0.80 (0.72 - 1.28) -1.4 0.77 (0.68 - 1.32) -1.6 C06 0.270 0.108 1.18 (0.75 - 1.25) 1.3 1.13 (0.75 - 1.25) 1.0 C07 2.398 0.120 1.32 (0.76 - 1.24) 2.4 1.36 (0.75 - 1.25) 2.6 C08 -0.961 0.124 1.20 (0.73 - 1.27) 1.4 1.14 (0.72 - 1.28) 1.0 C09 0.149 0.110 1.07 (0.75 - 1.25) 0.6 1.06 (0.75 - 1.25) 0.5 C10 -1.105 0.121 1.30 (0.75 - 1.25) 2.2 1.27 (0.73 - 1.27) 1.9 C11 -0.279 0.103 1.54 (0.75 - 1.25) 3.7 1.67 (0.74 - 1.26) 4.4 C12 -0.042 0.131 1.12 (0.74 - 1.26) 0.9 1.12 (0.74 - 1.26) 0.9 C13 -0.501* 1.66 (0.74 - 1.26) 4.3 1.65 (0.74 - 1.26) 4.2 *Fixed parameter; SE: standard error; CI: confidence interval

Note: Authors’ data

The raters exhibited significant differences in severity, χ2(12) = 723.1; p < .001, with the reliability of the separation index between raters

being very high (R = 0.985). The range of severity is 3.5 logits (λC07 = 2.4

vs. λC10 = -1.1), while the standard deviation of the latent distribution, expressed as the square root of the sample variance, is 1.6 logits. Even after eliminating rater C07 the differences are still large, as the distance between the other most severe raters (C02 and C04) and the most lenient (C10, C05 and C08) is around 1.9 logits. So, depending on the severity of the rater, the same student’s score may vary by more than one standard deviation of the latent distribution. The data do not indicate the existence of halo or central tendency effects among the raters. The lowest value of fit is for C05 (MNSQ-Uw = 0.80; MNSQ-W = 0.77) although it is much higher than the critical value of MNSQ = 0.5. Therefore we may discount over-consistency in the raters as a consequence of poor management of the rubric. On the contrary, the indices of fit indicate that three raters (C10, C11 and C13) had inconsistent handling of the rubrics. In those three cases MNSQ is

94 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation over 1.5, higher than the most lenient of Eckes’ (2005) criteria. In each case, the level of poor fit is small, so much so that when the criteria from Wolfe and McVay (2012) are applied, only the values from C13 seem to have poor fit, with their scores exhibiting greater variation than expected by the model.

Rubric effects

Table V shows the parameters of the cognitive processes (or grading criteria) and indicates that the raters had been able to discriminate between them. The overall differences between processes were significant, χ2(2) = 532.6; p < .001, and the reliability of the separation index between criteria was very high (R = 986).

TABLE V. Rubric effects

SE of dif- Unweighted fit Weighted fit Criteria Difficulty ficulty (Uw-fit) (W-fit) MNSQ (CI) T MNSQ (CI) T 1.11 1.10 Coherence 0.415 0.042 1.5 1.3 (0.86 – 1.14) (0.86 – 1.14) 1.03 1.05 Cohesion 0.931 0.045 0.4 0.6 (0.86 – 1.14) (0.86 – 1.14) 1.42 1.51 Presentation -1.345* 5.1 5.4 (0.86 – 1.14) (0.84 – 1.16) *Fixed parameter; SE: standard error; CI: confidence interval

Note: Authors’ data

The values indicate that the most difficult cognitive process was cohesion (λ = 0.9), while the easiest was appropriateness and presentation (λ = -1.3). This range is reasonable given that the evaluation standards underlying the latter process (persuasive-argument text, tidy, following margins etc.) are well consolidated in the majority of 12-year-old students. Nonetheless, this process seems to have a poor fit (MNSQ > 1.3), which

Revista de Educación, 386. October-December 2019, pp. 85-107 95 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

indicates that the raters did not make consistent use of the rubric for this process. The high MNSQ values indicate that this process is less discriminant than the other two, and that the raters tended to give grades which were at the extremes (0 points or 2 points). In particular the grades were unexpectedly biased high (2 points), which is why the MNSQ value is over 1.3 points. The initial principal component analysis indicated that the factorial weights of the Organization and Cohesion processes were double that of the appropriateness and presentation process. This would seem to indicate that the latter process, from a dimensional perspective of the scale, is evaluating different questions to the others. On removing the appropriateness and presentation scale, the fit of the model improves greatly, and even the values of poor fit for the raters disappear. This seems to indicate that the poor fit found in the raters could be due to a fault in the correct specification of the grading rubrics rather than inconsistent use of those rubrics.

Rater-added bias

The clearest rater effects, as previously stated, are the differences in rater severity. As raters were assigned to rating groups using a matrix design, it is easy to say that some groups have been stricter, and others more lenient. Figures I and II illustrate the results of not considering the effect of group severity on student grades. In the figures, each point indicates the position of a student, comparing their score expressed as points directly without correcting for the effect of the grading group (vertical axis) with the logit score for the same student, considering the severity of the group assigned to grade that student (horizontal axis). To make it easier to interpret the results, the means in both figures have been included for the classical scores (1.3 points) and in a transformed score, corrected for group severity (0.95 logits), dividing the cartesian space into four quadrants.

96 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

FIGURE I. Comparison of direct and transformed scores for students graded by TB03 and TB07 Direct score: not including grading group severity effects group not including grading score: Direct Transformed score: including the effect of grading group severity

Note: Authors’ data

FIGURE II. Comparison of direct and transformed scores for students graded by TB06 and TB12 Direct score: not including grading group severity effects group not including grading score: Direct Transformed score: including the effect of grading group severity

Note: Authors’ data

Revista de Educación, 386. October-December 2019, pp. 85-107 97 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Figure I compares the scores of grading groups 3 and 7, in which the

level of severity of the correctors was very similar (λTB03 = 0.013 and λTB07 = 0.006). The two sets of points appear to be superimposed, indicating that the students in the bottom-left quadrant, those with a classical score below the mean, also have below-average scores in the model which corrects for the grading group severity. Similarly, the points in the upper- right quadrant are those students with higher than average direct scores who also have higher than average results when grading group severity is taken into account. Figure II compares the results of two more varied grading groups: TB06 and TB12, which were the strictest and the most lenient of the groups respectively. Most students in TB06 are at or above average in classical scores, while in TB12 the situation is the opposite, most students are clearly punished by being assigned to a group which the four raters were collectively the strictest. However, when the effect of the grader is taken into account the situation changes dramatically. The red points in the top-left quadrant in figure II are those cases who have been clearly advantaged by the leniency of TB06, as in classical terms they have scored above the test average. However, scoring 1.3 in TB06 is not the same as in TB12, when the model corrects for grading group difficulty, students who score 1.3 points in the lenient grading group find themselves below the average in the transformed models (by approximately 0.3 logits), while students with similar scores in the stricter group score between 1 and 1.3 logits, in other words they are at or above the mean transformed score. Taking the cutoff points from the classical and corrected means (1.3 points and 0.95 logits respectively) to decide the students’ competency, we find that 41 of the 375 cases, or 15.4%, would have had their decision changed. In 29 cases students had above-average classical scores, but were below the average in transformed points. They had benefited from the leniency of the group they were assigned to. The 12 remaining cases (8 of whom were in TB12) were the opposite, the severity of the grading group put them below average in classical terms while their corrected scores were above average.

98 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Discussion and conclusions

One of the difficulties of high-stakes educational evaluations is ensuring objectivity and fairness in the scores. Objective and rigorous grading of writings and constructed item responses is a true challenge. Various methodological solutions are needed in order to achieve this objective. Firstly, systematic training needs to be given to the raters and secondly, appropriate linked block design needs to be used to assign raters to groups (Frey, et al., 2009; OECD, 2014). This study demonstrates the potential of balanced, incomplete designs to control the effect of assigning raters to groups. Fernández-Alonso & Muñiz (2011) indicated the advantages and disadvantages of using complete and incomplete designs. However the 13-block Youden square used in this study brings together the robustness and consistency of the Latin square with the balance and efficiency of balanced, incomplete designs. From a psychometric point of view, the MFRM models used exhibit enormous possibilities for carrying out integrated diagnostics of the grading rubrics, with significant results related to the four study objectives. With respect to the first objective, the interaction model between raters and cognitive processes (criteria) demonstrated the best fit, identifying differences between raters in interpreting the rubrics of the cognitive processes being evaluated. This result is in line with Adams and Wu (2010) and Eckes (2005), in which the more complex models had better fit than the more parsimonious principal effects model. In terms of the second objective, analysis with MFRM demonstrated its capacity to detect grading bias, finding wide variation in the levels of rater severity, estimated at more than one standard deviation in the scoring scale, even when discounting the effect of the most extreme raters. This data is in line with previous research indicating that between-rater differences are the norm and not the exception (Congdon & McQueen, 2000; Eckes, 2005; Lopes et al., 2015; Lunz et al., 1990; Prieto-Adánez, 2011; Prieto-Adánez & Nieto, 2014; Wolfe, 2004; Wolfe & McVay, 2012). Poor fit and inconsistent scores were identified in three of the thirteen raters, although according to some of the criteria in the literature (Wolfe & McVay, 2012) they would be within the expected limits of poor fit, except for one case. Nonetheless, we found indications that this anomaly may not be due so much to low intra-rater reliability, but rather to a structural problem with the rubric itself.

Revista de Educación, 386. October-December 2019, pp. 85-107 99 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

The analysis of the evaluation criteria rubrics (the third objective) indicated a mismatch in the process Appropriateness and Presentation, which was the least discriminant process and had the most polarized scores which were unexpectedly biased towards the top end of the scale. The exploratory analysis of the principal components indicated that the inclusion of this process could be problematic and that its rubrics should be reviewed, as when this analysis was removed, all of the model parameters (including rater fit) gave appropriate values. The literature has documented this kind of problem, either because the criteria are reiterative (Adams & Wu, 2010), or for a lack of consistency between them (Basturk, 2008; Lunz et al., 1990), which shows the possibilities offered by MFRM analysis for detecting design errors in grading rubrics. The final study objective was to demonstrate the practical advantage of MFRM, and in particular the possibility of having student scores corrected for severity in raters or rating groups. Previous research has shown that including rater-effects can result in significant variation in some students’ scores (Lunz et al., 1990; Wang & Yao, 2013). In our study, the data confirms that students with an identical score can be separated by approximately one logit when the model considers rating group severity. We also found that the result of the test in 15.4% of cases could change depending on whether the effect of rater-group severity is taken into account or not. This result is similar to Eckes (2005), who reanalyzed the results of the TestDAF (a language certification test in Germany) and estimated that 13.5% of the writings and 17.1% of the speaking tests would have had the result of the test changed (certified or not certified) had the rater-effect been taken into account. The data in this study have significant educational implications. On the one hand, they demonstrate the possibilities of MFRM models for identifying rater effects and possible defects in the definition of the variable to study or in grading rubrics. They also show that when differences in rater severity are taken into account, the results may vary, occasionally significantly. In Spain, there are various high-stakes exams such as university entrance exams and language certification in which the effects of rater severity will have an impact on student results. There is research looking at rater consistency in both of these types of exams (Amengual & Méndez García, 2012; Cuxart- Jardí, 2000; Cuxart- Jardí et al., 1997; Grau, Cuxart & Martí-Recober, 2002; Prieto-Adánez, 2011; Prieto-Adánez & Nieto, 2014; Suárez-Álvarez, González-Prieto,

100 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Fernández-Alonso, Gil & Muñiz, 2014), however, as far as we are aware, there is no research that has focused on the practical consequences to the student of having their grading assigned to raters who probably have varied levels of severity. Given the data provided by this study and bearing in mind the high impact of these types of tests on students’ academic and personal futures, it seems necessary to explore these processes of grading and scoring in more detail. It would also be important to examine the possible differential functioning of items induced by raters and rubrics (Gómez- Benito, Sireci, Padilla, Hidalgo & Benítez, 2018; Woitschach, Zumbo & Fernández-Alonso, 2019), as well as checking the generalization of the results to evaluations of students with different characteristics (Amor, Verdugo, Calvo, Navas & Aguayo, 2018). Besides the aforementioned analyses, one fundamental aspect is the development of systematic training and practice for evaluators in order to minimize the individual differences related to their grading and scoring strategies. In short, this means complementing training and practice with procedures for analyzing the results, using the most current, sophisticated psychometric and statistical models. One would hope that this pre-post combination would reduce to a minimum the errors in rigorous, objective student evaluation, an unavoidable ethical and deontological requirement This work is not without its limitations, the results should be taken with caution, and should not be universally generalized without more thought. There are three clear sources of error which may affect the results of this kind of work. Firstly, the type of evaluator, whose personal characteristics, training, background and involvement can vary from one to another. The subject being evaluated may also lead to restrictions that could affect the results. Finally, the design of the rubrics themselves may influence the results of the evaluations, and may for example, minimize or maximize the variance in the resulting scores.

References

Adams, R. & Wu, M. (2010). The analysis of rater effects. Recuperado de: https://www.acer.org/files/Conquest-Tutorial-3-RaterEffects.pdf Amengual, M. (2004). Análisis de la fiabilidad en las puntuaciones holísticas en ítems abiertos [Reliability analysis in holistic scores on

Revista de Educación, 386. October-December 2019, pp. 85-107 101 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

constructed response ítems]. Universidad Complutense de Madrid: Tesis doctoral no publicada. Recuperado de: http://biblioteca.ucm.es/ tesis/fll/ucm-t26663.pdf Amengual, M. & Méndez García, M. C. (2012). Implementing the oral English task in the spanish university admission examination: An international perspective of the language. Revista de Educación, 357, 105-127. Amor, A. M., Verdugo, M. A., Calvo, M. I., Navas, P. & Aguayo, V. (2018). Psychoeducational assessment of students with intellectual disability: professional-action framework analysis. Psicothema, 30, 39-45. doi: 10.7334/psicothema2017.175 Ato, M. & Vallejo, G. (2007). Diseños experimentales en psicología [Experimental designs in psychology]. Madrid: Pirámide. Baird, J. A., Hayes, M., Johnson, R., Johnson, S. & Lamprianou, I. (2013). Marker effects and examination reliability. A comparative exploration from the perspectives of generalizability theory, Rasch modelling and multilevel modelling (Ofqual/13/5261). Coventry: Office of Qualifications and Examinations Regulation. Recuperado de: https:// www.gov.uk/government/uploads/system/uploads/attachment_data/ file/378059/2013-01-21-marker-effects-and-examination-reliability.pdf Baird, J. A., Meadows, M., Leckie, G. & Caro, D. (2017). Rater accuracy and training group effects in Expert- and Supervisor-based monitoring systems. Assessment in Education: Principles, Policy and Practice, 24(1), 44-59. doi: 10.1080/0969594X.2015.1108283 Basturk, R. (2008). Applying the many-facet Rasch model to evaluate PowerPoint presentation performance in higher education. Assessment y Evaluation in Higher Education, 33(4), 431- 444. doi: 10.1080/02602930701562775 Bejar, I. I. (2012): Rater cognition: Implications for validity. Educational Measurement, 31(3), 2-9. Crisp, V. (2012). An investigation of rater cognition in the assessment of projects. Educational Measurement, 31(3), 10-20. Cochran, W.G. & Cox, G.M. (1974). Diseños experimentales [Experimental designs]. México: Trillas. (orig. 1957). Congdon, P. J. & McQueen, J. (2000). The stability of rater severity in large- scale assessment programs. Journal of Educational Measurement, 37(2), 163-178. doi: 10.1111/j.1745-3984.2000.tb01081.x

102 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Cuxart-Jardí, A. (2000). Modelos estadísticos y evaluación: tres estudios en educación. [Statistical models and assessment: three studies in education]. Revista de Educación, 323, pp. 369-394 Cuxart-Jardí, A., Martí-Recober, M. & Ferrer-Juliá, F. (1997). Algunos factores que inciden en el rendimiento y la evaluación en los alumnos de las pruebas de aptitud de acceso a la universidad [Some factors that affect the students’ performance in the university entrance exams]. Revista de Educación, 314, 63-88. Eckes, T. (2005). Examining rater effects in TestDaF writing and speaking performance assessments: A multi-faceted Rasch analysis. Language Assessment Quarterly, 2(3), 197-221. doi: 10.1207/s15434311laq0203_2 Eckes, T. (2009). Many-facet Rasch measurement. In S. Takala (Ed.), Reference supplement to the manual for relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment (Section H). Strasbourg, France: Council of Europe/Language Policy. Recuperado de: https:// rm.coe.int/1680667a23#search=eckes European Commission/EACEA/Eurydice (2009). National testing of pupils in Europe: Objectives, organization and use of results, Luxembourg: Publications Office of the European Union. doi: 10.2797/18294 Engelhard, G. (1992). The measurement of writing ability with a multi- faceted Rasch model. Applied Measurement in Education, 5(3), 171- 191. doi: 10.1207/s15324818ame0503_1 Engelhard, G. (1994). Examining rater errors in the assessment of written composition with a multi-faceted Rasch model. Journal of Educational Measurement, 31(2), 93-112. doi: 10.1111/j.1745-3984.1994.tb00436.x Fernández-Alonso, R. & Muñiz, J. (2011). Diseño de cuadernillos para la evaluación de las competencias básicas [Booklet designs for the evaluation of basic skills]. Aula Abierta, 39(2), 3-34. Frey, A., Hartig, J. & Rupp, A. A. (2009). An NCME instructional module on booklet designs in large-scale assessments of student achievement: Theory and practice. Educational Measurement: Issues and Practice, 28(3), 39-53. Gómez-Benito, J., Sireci, S., Padilla, J. L., Hidalgo, M. D. & Benítez, I. (2018). Differential item functioning: Beyond validity evidence based on internal structure. Psicothema, 30, 104-109. doi: 10.7334/ psicothema2017.183

Revista de Educación, 386. October-December 2019, pp. 85-107 103 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Grau, R. M., Cuxart, A. & Martí-Recober, M. (2002). La calidad en el proceso de corrección de las pruebas de acceso a la universidad: variabilidad y factores [Quality in the scoring process of university entrance exams: variability and factors]. Revista de Investigación Educativa, 20(1), 209- 224. Gyagenda, I. & Engelhard, G. (2009). Using classical and modern measurement theories to explore rater, domain, and gender influences on student writing ability. Journal of Applied Measurement, 10(3), 225-246. Jonsson, A. & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144. Kuo, S. A. (2007): Which rubric is more suitable for NSS liberal studies? Analytic or holistic? Educational Research Journal, 22(2), 179-199. Lallmamode, S. P., Mat Daud, N. & Abu Kassim, N. L. (2016). Writing Development and initial argument-based validation of a scoring rubric used in the assessment of L2 writing electronic portfolios. Assessing Writing, 30, 44-62. http://dx.doi.org/10.1016/j.asw.2016.06.001 Leckie, G. & Baird, J. (2011). Rater effects on essay scoring: A multilevel analysis of severity drift, central tendency, and rater experience. Journal of Educational Measurement, 48(4), 399-418. doi: 10.1111/j.1745- 3984.2011.00152.x Ley Orgánica 15/1999 de Protección de Datos de Carácter Personal [Organic Law 15/1999 on Protection of Personal Data]. Boletín Oficial del Estado de 14 de diciembre de 1999. Recuperado de: https://www. boe.es/buscar/doc.php?id=BOE-A-1999-23750 Ley Orgánica 8/2013 para la Mejora de Calidad Educativa [Organic Law 8/2013 for the Improvement of Educational Quality]. Boletín Oficial del Estado de 10 de diciembre de 2013. Recuperado de: https://www. boe.es/buscar/pdf/2013/BOE-A-2013-12886-consolidado.pdf Linacre, J. M., Engelhard, G., Tatum, D. S. & Myford, C. M. (1994) Measurement with judges: Many-faceted conjoint measurement. International Journal of Educational Research, 21(6), 569-577. doi: 10.1016/0883-0355(94)90011-6 Lopes Toffoli, S. F., de Andrade, D. F. & Bornia, A. C., (2015): Evaluation of open items using the many-facet Rasch model, Journal of Applied Statistics, doi: 10.1080/02664763.2015.1049938

104 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Lunz, M. E. & Stahl, J. (1990). Judge consistency and severity across grading periods. Evaluation and the Health Professions, 13(4), 425- 444. doi: 10.1177/016327879001300405 Lunz, M. E., Wright, B. D. & Linacre, J. M. (1990). Measuring the impact of judge severity on examination scores. Applied Measurement in Education, 3(4), 331-345. doi: 10.1207/s15324818ame0304_3 Mahmood, O., Dagnæs, J., Bube, S., Rohrsted, M. & Konge, L. (2017). Nonspecialist raters can provide reliable assessments of procedural skills. Journal of Surgical. Recuperado de: http://dx.doi.org/10.1016/j. jsurg.2017.07.003 McNamara, T. F. (1996). Measuring second language performance. London: Longman Ministerio de Educación, Cultura & Deporte (2016). Pruebas de la evaluación final de Educación Primaria. Curso 2015-2016 [Tests of the final evaluation of Primary Education. Course 2015-2016]. Madrid: Instituto de Evaluación Myford, C. M. & Wolfe, E. W. (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4, 386-422 Myford, C. M. & Wolfe, E. W. (2004). Detecting and measuring rater effects using many-facet Rasch measurement: Part II. Journal of Applied Measurement, 5, 189-227 Organisation for Economic Co-operation and Development [OECD]. (2014). PISA 2012 Technical Report. Paris: OECD Publishing. Recuperado de: https://www.oecd.org/pisa/pisaproducts/PISA-2012- technical-report-final.pdf Park, T. (2010). An investigation of an ESL placement test of writing using multi-faceted Rasch measurement. Teachers College, Columbia University Working Papers in TESOL and Applied Linguistics, 4(1), 1-19 Prieto-Adánez, G. (2011). Evaluación de la ejecución mediante el modelo Many Facet Rasch Measurement [Performance assessment using the Many-Facet Rasch Measurement]. Psicothema, 23, 233-238 Prieto-Adánez, G. (2015). Análisis de un test de desempeño en expresión escrita mediante el modelo de MFRM [Analysis of a writing test with the MFRM model]. Actualidades en Psicología, 29(119), 1-17. http:// dx.doi.org/10.15517/ap.v29i119.19822

Revista de Educación, 386. October-December 2019, pp. 85-107 105 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Prieto-Adánez, G. & Nieto, E. (2014). Analysis of rater severity on written expression exam using Many Faceted Rasch Measurement. Psicológica, 35, 385-397 Saal, F. E., Downey, R. G. & Lahey, M. A. (1980). Rating the ratings: Assessing the psychometric quality of rating data. Psychological Bulletin, 88(2), 413-428 Shackleton, C. (2018). Linking the University of Granada CertAcles listening test to the CEFR. Revista de Educación, 381, 35-60. doi: 10.4438/1988-592X-RE-2017-381-380 Suárez Álvarez, J., González-Prieto, C., Fernández Alonso, R., Gil, G. & Muñiz, J. (2014). Psychometric assessment of oral expression in English language in the University Entrance Examination. Revista de Educación, 364, 93-118. doi: 10.4438/1988-592X-RE-2014-364-256 Sudweeks, R. R., Reeve, S. & Bradshaw, W. S. (2005). A comparison of generalizability theory and many facet measurement in an analysis of college sophomore writing. Assessing Writing, 9, 239-261. Suto, I. (2012). A critical review of some qualitative research methods used to explore rater cognition. Educational Measurement, 31 (3), 21-30. Wang, Z. & Yao, L. (2013). The effects of rater severity and rater distribution on examinees’ ability estimation for constructed-response items. ETS Research Report Series, i-22. doi:10.1002/j.2333-8504.2013.tb02330.x Wolfe, E. W. (2004). Identifying rater effects using latent trait models. Psychology Science, 46(1), 35-51. Wolfe, E. W. & McVay, A. (2012). Application of latent trait models to identifying substantively interesting raters. Educational Measurement, 31 (3), 31-37. Woitschach, P., Díaz-Pérez, C., Fernández-Argüelles, D., Fernández- Castañón, J., Fernández-Castillo, A., Fernández-Rodríguez, L., ... & Fernández-Alonso, R. (2018). Efectos del corrector en las evaluaciones educativas de alto impacto. [Rater effects in high-impact educational assessments]. REMA Revista Electrónica de Metodología Aplicada, 23(1), 12-27. Woitschach, P., Zumbo, B. D. & Fernández-Alonso, R. (2019). An ecological view of measurement: Focus on multilevel model explanation of differential item functioning. Psicothema, 31(2), 194-203. doi:10.7334/ psicothema2018.303

106 Revista de Educación, 386. October-December 2019, pp. 85-107 Received: 16-11-2018 Accepted: 21-06-2019 Fernández-Alonso, R., Woitschach, P., Muñiz, J. Rubrics do not neutralize rater effects: A many-faceted Rasch model estimation

Wu, M. L., Adams, R. J., Wilson, M. R. & Haldane, S. A. (2007). ACER ConQuest 2.0: generalised item response modelling software. Camberwell, Victoria: Australian Council for Educational Research.

Contact address: Rubén Fernández-Alonso. Consejería de Educación y Cultura del Gobierno del Principado de Asturias y Universidad de Oviedo, Facultad de Formación del Profesorado y Educación, Departamento de Ciencias de la Educación. Calle Aniceto Sela, 1, 33005 Oviedo, Asturias. E-mail: [email protected]

Revista de Educación, 386. October-December 2019, pp. 85-107 107 Received: 16-11-2018 Accepted: 21-06-2019

Scientific production on gamification in education: a scientometric analysis

Producción científica sobre gamificación en educación: un análisis cienciométrico

DOI: 10.4438/1988-592X-RE-2019-386-429

M.ª Elena Parra-González Adrián Segura-Robles Universidad de Granada

Abstract Gamification is an emerging methodology that consists of using games or game mechanics in non-ludic contexts, providing motivation for students in an educational context. This phenomenon has caused it not only to be used in classrooms as concrete experiences but also in the field of science, where there has been a proliferation of research on educational gamification in recent years. This growing interest has led to this study, which consists of a scientometric analysis of the publications indexed in the Web of Science (WoS) impact databases on the emerging methodology of gamification in education. The following categories were surveyed: Education & Educational Research, Education Scientific Disciplines, Social Sciences Interdisciplinary and Psychology Educational. The sample comprised all scientific productions (articles and conference communications) from their first appearance until 2017, for a total of 345 documents. Scientometric analysis focused on analyzing chronological productivity, productivity according to the type of document and according to the publisher, the language and the distribution of productions according to Bradford’s and Lotka’s laws. The results show that quality scientific production has entered an exponential growth phase, which evinces an increased interest of this methodology in the classroom. The number of productions has declined in the last year, but the scientific quality of these results confirms the importance of gamification in the field. Last, it should be noted that the majority of publications are in English, as it is a global scientific language.

Revista de Educación, 386. October-December 2019, pp. 109-131 109 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

Key words: educational games, scientometrics, bibliometry, educational innovation, educational research.

Resumen La gamificación es una metodología emergente que consiste en utilizar juegos o mecánicas de juego en contextos no lúdicos, proporcionando en educación la motivación de los alumnos. Este fenómeno ha provocado que no sólo se utilice en las aulas como experiencias concretas, sino que en el campo científico estén proliferando las investigaciones sobre gamificación educativa en los últimos años. Este creciente interés ha provocado esta investigación, que consiste en un análisis cienciométrico de las publicaciones indexadas en las bases de datos de impacto Web of Science (WoS) sobre la metodología emergente de gamificación en Educación. Se han tenido en cuenta las categorías: Education & Educational Research, Education Scientific Disciplines, Social Sciences Interdisciplinary y Psychology Educational. La muestra ha estado compuesta por todas las producciones científicas (artículos y comunicaciones en congresos) desde la primera aparición hasta 2017, con un total de 345 documentos. El análisis cienciométrico se ha centrado en analizar la productividad cronológica, la productividad según el tipo de documento, según la editorial, el idioma y la distribución de producciones según las leyes de Bradford y Lotka. Los resultados demuestran que las producciones científicas de calidad se encuentran en fase de crecimiento exponencial, con lo que se demuestra el incremento del interés de esta metodología en las aulas, han descendido en el último año la cantidad de producciones, pero aumenta la calidad científica de las mismas, lo que confirma la importancia de la gamificación en el ámbito. Destacar por último quela mayoría de publicaciones son en inglés, como idioma científico mundial.

Palabras clave: juegos educativos, cienciometría, bibliometría, innovación educativa, investigación educativa

Introduction

Gamification represents a powerful tool to help motivate students in class and thus facilitate the process of teaching and learning. It consists of using elements, designs or game structures in non-ludic contexts (Attali and Arieli-Attali, 2015; Bruder, 2015; Dale, 2014; Deterding, 2012; Deterding, Dixon, Khaled, Nacke, 2011; Deterding, Sicart, Nacke, O’Hara and Dixon, 2011; González et al., 2016; Hanus and Fox, 2015; Kapp,

110 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

2012a; Koivisto and Hamari, 2014; Powers et al., 2013; Whitton and Moseley, 2010). There has been a recent increase in this methodology in the classroom to motivate students (Banfield and Wilkerson, 2014). Gamification is thus presented as an emerging and innovative resource, with positive effects on the teaching-learning process (Denny, 2013). Piaget (1962), early on, defended the use of game elements in education, emphasizing that the use of games helps to create worlds of imagination. Some years later, studies by Malone (1980) and Malone and Lepper (1987) began to highlight the use of games as powerful learning tools. A review of the literature tells us that the use of games or their elements in the classroom can occur at all levels, from elementary school education (Su and Cheng, 2015) though high school education (Giannakos, 2013) to higher education (Dib and Adamo-Villani, 2014). Foncubierta and Rodríguez (2016) note that a game is a finished product and that it is something concrete, while gamification is based on didactic content; they define it as an activity seasoned with the elements and spirit of play (Seaborn and Fels, 2015). According to Simões, Díaz and Fernández (2013), with gamified activity, students learn as if they were playing a game, not playing specific games. Thus, it would appear that students would learn effortlessly. Through gamification, effort is valued and rewarded with badges, not only by achievement, as in the traditional teaching methodology. Brewer et al. (2013) studied the use of gamification in children aged 5 to 7 years, concluding that gamification implies a significant increase in finished tasks. Wang (2015) also states that the use of games involves students in highly elevated percentages. Mekler et al. (2013, 2017) also found that assigning points, levels and scoreboards increases the performance by the user but has no effect on the user’s competence, autonomy or intrinsic motivation. Many teachers use games in the classroom, but some of these professionals have recently turned their classroom into a game (Kapp, 2012b). Furthermore, the games usually allow students to play repeatedly, so that they have the freedom to experiment again and again, learning from their mistakes without fear (Lee and Hammer, 2011). Although numerous authors consider this methodology innovative and attractive in motivating students and facilitating their learning,

Revista de Educación, 386. October-December 2019, pp. 109-131 111 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

Martí-Parreño, Seguí-Mas and Seguí-Mas (2016) state that although in recent years there has been increased academic interest in gamification, teachers’ attitudes and the actual use of gamification is not a well- researched field. Therefore, they conducted an exploratory study of the use of gamification in higher education and found that only 11.3% of teachers use this methodology in their classes, although the remainder sees it as highly positive. The latest trends point to the use of digital games and video games as another means of acquiring knowledge (Asensio et al. 2014; Contreras- Espinosa, 2016; Gallego, Villagrá, Satorre, Compañ, Molina and Lorens, 2014). Even today, companies have implemented gamification to improve the experiences of their employees (Schacht and Schacht, 2012). As we see, this methodology has been implemented in a wide range of fields, such as marketing and education, to improve the experience and commitment of users (Deterding, Sicart, Nacke, O’Hara and Dixon, 2011). Following this line of thought, Trujillo (2014) states that through ICT in the classroom, we can create digital artifacts in teaching-learning processes to contextualize them. Thus, gamification and the game itself would take on a new importance in the classroom by creating more significant learning, gains in space and importance in education. As mentioned earlier, the role of video games and ICT is very important because students will be able to play games later, sometimes even from home, and will be able to see the rankings, etc. However, we must also take into account the social component of games. Students will socialize through games; at other times, they will have to be more competitive but will also work cooperatively and solve problems together. Teachers should be careful, as the competitiveness factor, or sometimes the roles taken on within these dynamics, could have negative effects (Christy and Fox, 2014). On the other hand, one must be cautious as to how students are invited or encouraged to participate because students do not like “compulsory fun” either (Mollick and Rothbard, 2014). The extensive literature reviewed, therefore, clearly shows that the introduction of game mechanics with a concrete objective in the classroom provides significant learning and that their use has been shown to increase learning (De-Marcos, Domínguez, Saenz-de-Navarrete and Pagés, 2014; Gee, 2007). Although ludic activities have often been used in some classes, the meaning and structure of gamification goes further and provides greater benefits; in

112 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis addition, as seen in the literature review, there has been an increase in the use of this powerful tool in the classrooms over the past few years. While the concrete experiences of teachers with this educational innovation methodology were limited to classrooms, there is now increased interest by researchers and scientific studies. This is why, after detecting that there is no study investigating, analyzing and researching gamification, we have conducted this study to analyze the scientific work and numerous studies on this topic that have been of such great interest in recent years. Thus, we present a scientometric analysis of articles and scientific contributions on gamification in the educational field indexed in the Web of Science (WoS) impact databases.

Objectives

The main objective of this study is to scientometrically analyze production on gamification in education from its first appearance until 2017. The specific objectives are as follows: ■ To analyze the chronological productivity of gamification in educational categories; ■ To analyze productivity according to the type of document on gamification in educational categories; ■ To analyze the institutional productivity of gamification in educational categories; ■ To analyze the editorial productivity of gamification in educational categories; ■ To analyze the productivity of gamification in educational categories according to the language of publication; ■ To analyze the adjustment of production according to the scientific productivity zonal law; and ■ To analyze personal productivity on gamification in educational categories.

Method

The scientometric method is a scientific discipline that, through analysis, offers some indicators to analyze the progress and current scientific status

Revista de Educación, 386. October-December 2019, pp. 109-131 113 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

of a topic (Fernández-Cano and Bueno, 1999). It has been used by other authors in their educational research and has been recognized in the field (Gutiérrez-Braojos, Martín-Romera, Casasempere and Fernández- Cano, 2015). The method used to perform this scientometric analysis is adapted from the one used by Michán and Muñoz-Velasco (2013), which consists of five steps: recovery, migration, analysis, visualization and interpretation. In this adaptation, the first two steps are performed only once (as in the original), while the last three steps, analysis, visualization and interpretation, are repeated for each objective. The work shaping each step is as follows: ■ Recovery: selection of sources and resources, search and selection; ■ Migration: extraction, loading, cleaning and data processing; ■ Analysis: scientometric analysis, consultations, quantitative treatment (scientometric and bibliometric indicators); ■ Visualization: parameters, identification; and ■ Interpretation: description, comparison and contextualization.

Samples

The study sample comprises scientific productions (articles and conference papers) on gamification in education indexed in the Web of Science (WoS) in education fields (from the first production in 2011 to 2017) included under these categories: Education and Educational Research, Education Scientific Disciplines, Social Sciences Interdisciplinary and Psychology Educational. Therefore, the sample includes a total of 345 scientific productions (articles and conference papers) on educational gamification recovered from the databases SCI-EXPANDED, SSCI, CPCI-S, CPCI-SSH, ESCI.

Procedure

The indicators for the study variables focus on chronological production, production according to the type of document, institutional production, editorial production, production by language and distribution according to the most used scientific production laws.

114 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

The information search began with a first search to determine the number of scientific productions on educational gamification in the WoS in the educational area categories mentioned above. The next step was to decide the specific objectives and categories for the information search. All productions were then extracted and organized in a data matrix using SPSS version 25. The fundamental bibliographic data of the WoS were examined, such as the type of document, year, authors, institution, journal title, publisher, language and citations received. The data were analyzed (both in SPSS and EXCEL) and interpreted to present the results.

Results

To respond to both the general objective and the specific objectives, the results are presented below. To facilitate reading the data, the analyses are presented together with the interpretation of the data.

Chronological productivity

As seen, for the 2011-2017 period, we found a total of 345 indexed documents, including articles and conference papers. The first publication we found is dated 2011. The average for the period stands at 49.28 with a deviation of 43.94. For the temporary coverage analyzed, we found three important periods (Figure I). The first (2011-2013) is characterized by limited production that accounts for only 5.2% of the total production of documents. In the second (2014-2016), production increases, representing 72.1% of total production. Last, we observe the beginning of a third period (2017-onwards) in which we notice a significant 22.6% decrease in the number of publications.

Revista de Educación, 386. October-December 2019, pp. 109-131 115 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

FIGURE I. Chronological productivity during 2011-2017

Source: the authors

One possible cause for this sharp decrease is the increase in the rigor and quality of the publications; therefore, one must analyze the phenomenon from a perspective based on the type of document in order to understand the observation. In terms of chronological productivity according to the type of document, there is an increase in the scientific rigor of productions, especially in 2016, when the number of conference papers begins to decrease (91 in 2016 and 43 in 2017) compared to an increase in articles (29 in 2016 and 32 in 2017). This implies, in the first case, a decrease of -52.75% for papers and an increase of 10.34% for articles during the key period 2016-2017.

116 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

FIGURE II. Chronological productivity according to type of document

Source: the authors

In view of these results and given the results in Figure II, Price’s Law of exponential growth is not fulfilled due in large part to the fact that it takes into account time spans in the form of five-year periods or decades and that gamification is a relatively recent construct (dating only to 2010). However, we can see a decrease in papers and an increase in articles, which shows a fairly clear goal: improving the scientific quality of published research.

Institutional Productivity

According to the data classified according to the document’s institution of origin, we find that scientific production is focused on specific entities, both for articles and for papers. For articles, production is concentrated at 6 producing universities. As shown in Table I, being a highly productive institution does not ensure a greater impact, as in the case of Shu-te University, which has 3.96% of the total production but an impact of only 14.25, while the University of Alcalá, with a lower production, has the

Revista de Educación, 386. October-December 2019, pp. 109-131 117 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

largest number of citations received with a calculated impact of 44.33, the highest among all institutions.

TABLE I. Most relevant institutions based on article productivity

Cita- Impact Institution Docs Percentage tions index * Shu-te University 4 3.96% 57 14.25 National Yunlin University of 3 2.97% 57 19 Science Technology Universidad Europea Valencia 3 2.97% 18 6 University of Alcalá 3 2.97% 133 44.33 University of la Laguna 3 2.97% 3 1 UOC Open University of Catalonia 3 2.97% 1 0.33

Source: the authors. * = Calculated as Appointments/Docs

For conference papers, we found 4 leading production institutions. As shown in Table II, the production of papers exceeds that of articles, with the Universiti Teknologi Malaysia showing 6 indexed documents; however, contrary to what happens with documents published with a higher scientific rigor (such as articles), the citations generated and the calculated impact are far from being considered relevant to the current academic community.

TABLE II. Most relevant institutions based on conference paper productivity

Impact Institution Docs Percentage Citations index * Universiti Teknologi Malaysia 6 2.41% 1 0.16 Lublin University of Technology 5 2% 8 1.5 European University of Valencia 5 2% 2 0.4 University West London 5 2% 2 0.4

Source: the authors. * = Calculated as Appointments/Docs

118 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

Editorial productivity

Our analysis revealed a total of 65 publishers that brought together all publications on gamification during the period analyzed (Table III). Of these, 8 publishers stand out, with a total of 249 documents, that is, 72.12% of the total production on gamification. Likewise, we observed no direct relationship between the number of publications and the citations received. ELSEVIER, with 13 documents, lacks the greatest number of citations compared to IATED, which, despite having 106 documents on gamification, received a total of only 30 citations.

TABLE III. Most relevant publishers according to productivity

Publisher Docs Percentage Citations Impact index * IATED 106 30.72% 30 0.28 ACAD 52 15.07% 66 1.26 IEEE 47 13.62% 228 4.85 ELSEVIER 13 3.76% 371 28.53 IGI GLOBAL 10 2.89% 15 1.5 ASSOC COMPUTING 10 2.89% 10 1 MACHINERY TEMPUS 7 2.02% 16 2.28 WILEY 4 1.15% 67 16.75

Source: the authors. * = Calculated as Citations/Docs

Productivity by language

From the total of all documents identified, we note that documents on gamification are published in 4 languages: English, Spanish, French and Portuguese. Table IV shows that English is the language most commonly used, with 76.47% (65) of the total contributions, followed by Spanish, with 16.47% (14), and French and Portuguese, combined, below 10%. For conference papers, this percentage is even more drastic and significant, with English as the main language in 99.59% (246) of the total contributions.

Revista de Educación, 386. October-December 2019, pp. 109-131 119 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

TABLE IV. Articles and conference papers published according to language

Language Articles Total% Communications Total% English 65 76.47% 246 99.59% French 2 2.35% - - Portuguese 4 4.70% - - Spanish 14 16.47% 1 0.40%

Source: the authors.

Zonal distribution according to Bradford’s law

As has been done thus far and to make a clear differentiation, the results are shown in the two large groups analyzed. In this case, to find the journals and conference papers with the greatest producers of scientific activity related to gamification, the Bradfordian distribution or division by Bradford zones is used. For Bradford, if the analyzed producers are sorted in decreasing order, we can see a number of journals or communications in “nucleus” conferences that accumulate the largest amount of information and several zones that contain a similar number of documents produced that follow, approximately, a mathematical progression approximated to the form 1:n:n2... (Bradford, 1934, 1948). Today, as recent research shows (Mangas-Vega, Gómez-Díaz and Cordón-García, 2016), this continues to be a renowned method of journal article or conference paper classification in the social sciences (Palomo, Figueroa-Domecq and Laguna, 2017) and health sciences (Thompson and Walker, 2015). In our case, to observe the distribution of producing journals, we use Bradford’s classic zonal classification, opting for a total of 3 zones: zone 1 or the nucleus and two other zones. Table V shows that 38% (32) of the articles produced belong to a very delimited nucleus comprising, in this case, 7 journals. We also found a second zone, amounting to 29% (25) of them in approximately 12 journals, and a third zone with 32% (27) comprising 27 journals. In this case, the progression obtained, n = 1.98, is quite close to the one proposed in Bradford’s original model. That is, we can conclude that even if we do not follow a quadratic geometric progression (n = 2) such as the one proposed by Bradford, it is very close

120 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis to it. The journals that are part of the nucleus include the Bulletin of the Technical Committee on Learning Technology, Computers & Education and Eurasia Journal of Mathematics Science and Technology Education, all of them focused more on the technological field than purely on education.

TABLE V. Distribution of production journals according to Bradford’s zones

% Ar- % ar- Bradford Zones Journals jour- ticles ticles Multiplier nals Zone 1 (Nucleus) 7 15% 32 38% - Zone 2 12 26% 25 29% 1.71 Zone 3 27 59% 27 32% 2.25 TOTAL 46 100% 85 100% n =1.98 Magazines that are part of the nucleus Journals Nº of articles Bulletin of the Technical Committee on Learning Technology 5 Computers & Education 5 Eurasia Journal of Mathematics Science and Technology Education 5 Journal of e-learning and Knowledge Society 5 Electronic Journal of e-learning 4 International Journal of Emerging Technologies in Learning 4 International Journal of Game-based Learning 4

Source: the authors.

In the same way, the most productive conferences are divided by the chosen diachronic system. Table VI shows that 33% (74) of papers presented on gamification are found in two main conferences. This is followed by a second zone, comprising 7 conferences totaling 32% (71) of papers and a third zone that includes 50 different conferences with 35% (78) productivity. Contrary to what occurs with journal articles, the calculated distribution does not follow a quadratic mathematical progression. In this regard, two producing conferences stand out: ICERI

Revista de Educación, 386. October-December 2019, pp. 109-131 121 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

(International Conference of Education, Research and Innovation) and EDULEARN (International Conference on Education and New Learning Technologies), both focused on the field of education.

TABLE VI. Distribution of producing conferences according to Bradford’s zones

Brad- Accu- Accu- ford Communi- Zones Conference mula- mula- Multi- cations ted% ted% plier

Zone 1 (Nucleus) 2 3% 74 33% - Zone 2 7 12% 71 32% 3.5 Zone 3 50 85% 78 35% 7.14 TOTAL 59 100% 223 100% n=5.32 Conferences that are part of the nucleus Nº of communi- Conference cations International conference of education, research and innovation (ICERI). 40 EDULEARN: international conference on education and new learning 34 technologies.

Source: the authors.

Distribution of producers according to Lotka’s law

With respect to the authors, we verified if their distribution according to 2 their productivity conforms to the original model An= A1 / n proposed by m Lotka (1926) or fits a different model n=A A1/ n , where An is the number

of authors with n signatures, A1 is the number of authors with 1 signature and nm is the number of signatures squared. For Lotka, there are very few authors who publish the majority of the studies, i.e., for high productivity

(n), the number of authors (An) is low because both variables are inversely proportional. According to their estimates, the scientific literature tends to establish a progression of m = 2 in the majority of cases. In the rest of the situations in which this distribution is not met, it proposes the logarithmic function , where the intersection of the line equals the value of m adapted to the case studied.

122 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

The observed and expected calculations are shown following Lotka’s traditional theory (n2) in addition to the theoretical model adapted to the results obtained specifically in this study (Table VII). In addition, following what Lotka proposes, the number of elite authors is calculated using the square root of the total number of authors.

TABLE VII. Observed and expected results for Lotka’s traditional version and the calculated version (articles)

Studies Authors AI* AII** 1 188 188 188 2 8 47 13 3 2 21 3 4 1 12 1 Total 199 Most productive authors * Author Studies Su, Chung-Ho 4 Marti-Parreno, José 3 De-Marcos, Luis 3

Source: authors’ own elaboration * AI= Authors expected according to Lotka’s Law (m = 2), ** AII = Authors expected according to the adapted model (m = 3.8)

As observed, we cannot state that the results present a traditional distribution as proposed by the author, but in our case, the distribution followed by the data are equivalent to m=3.8 or, the equivalent, n3.8. The most productive authors, or “elites,” are those with 3 or more published articles related to gamification. For conference papers, the same procedure as previously performed is followed, and the results are shown in Table VIII. As observed, the distribution of the papers presented does not follow Lotka’s original distribution; following the previous logarithmic representation, we obtain a value of m = 3.6 for the sample analyzed. The most productive authors are those with 6 or more studies.

Revista de Educación, 386. October-December 2019, pp. 109-131 123 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

TABLE VIII. Observed and expected results for Lotka’s traditional version and the calculated version (papers)

Studies Authors AI* AII ** 1 549 549 549 2 48 137.25 45.27 3 7 61 10.51 5 2 21.96 1.67 6 4 15.25 0.86 Total 610 Most productive authors * Author Studies Abdullah, Zaleha 6 Zaid, Norasykin Mohd 6 Marti-Parreno, Jose 6 Sanmugam, Mageswaran 6

Source: authors’ own elaboration * AI= Authors expected according to Lotka’s Law (m = 2), ** AII = Authors expected according to the adapted model (m = 3.6)

Discussion and Conclusions

Our primary aim was to carry out the first study on the production and consumption of gamification in the education field using its indexing occurrences in the SCI-EXPANDED, SSCI, CPCI-S, CPCI-SSH, ESCI, and SECT databases, from the first indexed publication on the topic (2011) to 2017, to analyze the state and evolution of a topic that is booming in classrooms. Other studies, such as those of Seaborn and Fels (2015), use a meta-synthesis approach to present a comprehensive understanding and consensus on the conceptualization of a study object, describing, comparing and contrasting a matrix of sources on the topic that can be qualitative, quantitative or of mixed nature (Jensen and Allen, 1996; Heyvaert, M., Maes, B. and Onghena, 2013). We observe that productions with more scientific rigor are increasing, which confirms findings by Martí-Parreño, Seguí-Mas and Seguí-Mas (2016) that recent years have seen an increase in academic interest in

124 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis gamification. This interest is supported by the 2018 Report on the state of the educational system for the 2016/2017 academic year by the Ministry of Education and Vocational Training, which indicates that the second most requested course by teachers is gamification in the classroom, which supports the growing interest of teachers in this methodology. Our scientometric analysis has focused on analyzing chronological productivity, productivity according to the type of document, according to the publisher, the language and distribution of productions according to Bradford and Lotka. We can mainly draw the following conclusions: ■ A decrease in the number of publications has been detected in the last year because their quality increases. That is, there is an exponential growth in their quantity and scientific rigor, thus reducing the number of conference papers and increasing the number of articles. This differentiation in terms of rigorousness is found in Miguel (2012), who distinguished between a conference paper and the publication of articles in specialized journals, where there is an evaluation process of the quality of the articles to be published, while conference papers have less academic rigor because although there is an update, training and exchange of information between professionals and scientists, sometimes these papers are published as conference proceedings, which may or may not undergo a review process that guarantees their quality. Although this distinction cannot be made lightly, it depends on other variables, such as the field or type of conference. ■ Regarding scientific production, one notable difference can be observed in terms of quality, that is, the greater is the scientific rigor, the lower the quantity of contributions and the greater their impact (measured as citations received and impact), an effect observed in this study. ■ There are institutions that accumulate a greater number of generated scientific documents, both articles and papers, which, in addition, bring together a larger number of citations and impact. Foreign universities stand out over national ones in this respect. ■ As for production by language, we can say that it is not surprising that the majority of contributions are in English because, as indicated by Price (1971, 1986), more than half of the scientific and philosophical literature is published in this language. This is observed for both papers and articles.

Revista de Educación, 386. October-December 2019, pp. 109-131 125 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

■ With respect to the production distributed by Bradford’s zones in conference papers and that does not follow a mathematical quadratic progression, this may be due to the dynamics of conferences, which tend to revolve around specific themes, and this may cause this type of distortion of results. ■ Among authors, there is also a clear differentiation between those who focus on more rigorous research (articles) and those whose contributions are published in conference proceedings. This article represents the first scientometric study of educational gamification. Given the interest in the subject in education, the possibility of carrying out another study of similar characteristics over a time period of no less than five years to analyze the interest in and the production of the topic during this period is considered. With respect to an approach for future studies, we propose analyzing gamification by education level, separately analyzing gamification in elementary schools, high schools or in higher education, to determine its influence by level and hoping that the literature will follow the expected growth analyzed in this research.

References

Asensio, J. J.; Mora, A. M.; Fernández, A. J.; García-Sánchez, P.; Merelo, J. J. y Castillo, P. A. (2014). Programar: aprendiendo a programar usando videojuegos como metáfora para visualización de código. ReVisión. Revista de Investigación en Docencia Universitaria de la Informática, 7(2), 93–103. Attali, Y. y Arieli-Attali, A. (2015). Gamification in assessment: do points affect test performance? Computers and Education, 83(1), 57-63. Recuperado de http://doi.org/10.1016/j.compedu.2014.12.012. Banfield, J., y Wilkerson, B. (2014). Increasing student intrinsic motivation and self-efficacy through gamification pedagogy.Contemporary Issues in Education Research, 7(4), 291-298. Bradford, S. C. (1934). Sources of information on specific subjects. Engineering, 137, 85-86.

126 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

Bradford, S. C. (1948). Documentation. Londres: Crosby Lockwood. Brewer, R., Anthony, L., Brown, Q., Irwin, G., Nias, J. y Tate, B. (2013). Using gamification to motivate children to complete empirical studies in lab environments. 12th International Conference on Interaction Design and Children, New York, NY, pp. 388-391. Bruder, P. (2015). Game on: gamification in the classroom. Education Digest, 80(7), 50-56. Christy, K. R. y Fox, J. (2014). Leaderboards in academic contexts: A test of stereotype threat and social comparison explanations for women’s math performance. Computers & Education, 78, 66-77. Recuperado de: http://dx.doi.ogr/10.106/j.compedu.2014.05.005. Contreras-Espinosa, R. S. (2016). Juegos digitales y gamificación aplicados en el ámbito de la educación. RIED. Revista Iberoamericana de Educación a Distancia, 19(2), 27–33. Recuperado de: http://dx.doi. org /10.5944/ried.19.2.16143. Dale, S. (2014). Gamification: making work fun, or making fun of work? Business Information Review, 31(2), 82-90. Recuperado de: http://doi. org/10.1177/0266382114538350. De-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J. y Pagés, C. (2014). An empirical study comparing gamification and Social networking on e-learning. Computers & Education, 75, 82-91. Recuperado de: http:// dx.doi.ogr/10.1016/j.compedu.2012.12.020. Denny, P. (2013). The effect of virtual achievements on student engagement. En Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI13’. ACM, París: Francia, pp. 763– 772. Deterding, S. (2012). Gamification: designing for motivation.Interactions, 19(4), 14-17. Deterding, S., Dixon, D., Khaled, R., Nacke, L. (2011). From game design elements to gamefulness: defining gamification. In Proceedings of the 15th International Academic MindTrek Conference, pp. 9–15. Deterding, S., Sicart, M., Nacke, L., O’Hara, K. y Dixon, D. (2011). Gamification: using game-design elements in non-gaming contexts. In Proceedings of the 2011 Annual Conference of Human Factors in Computing Systems, pp. 2425-2428. Recuperado de: http://dx.doi. org/10.1145/1979742.1979575. Dib, H., y Adamo-Villani, N. (2014). Serious Sustainability Challenge Game to Promote Teaching and Learning of Building Sustainability.

Revista de Educación, 386. October-December 2019, pp. 109-131 127 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

Journal of Computing in Civil Engineering, 28, A40140071- A401400711. Fernández-Cano, A. y Bueno, A. (1999). Synthesizing scientometric patterns in Spanish educational research. Scientometrics, 46(2), 349- 367. Foncubierta, J. M., Rodríguez, C. (2016). Didáctica de la gamificación en la clase de español. Editorial Edinumen. Recuperado de: https://www. edinumen.es/spanish_challenge/gamificacion_didactica.pdf. Gallego, F. J.; Villagrá, C. J.; Satorre, R.; Compañ, P; Molina, R. y Lorens, F. (2014). Panorámica: serious games, gamificación y mucho más. Re Visión. Revista de Investigación en Docencia Universitaria de la Informática, 7(2), 13–23. Gee, J. P. (2007). What video games have to teach us about learning and literacy: Revised and updated edition. New York, NY: Macmillan. Giannakos, M. N. (2013). Enjoy and learn with educational games: Examining factors affecting learning performance. Computers & Education, 68, 429–439. González, C.S., Gómez, N., Navarro, V., Cairós, M., Quirce, C., Toledo, P. y Marrero-Gordillo, N. (2016). Learning healthy lifestyles through active videogames, motor games and the gamification of educational activities. Computers in Human Behavior, 55(A), 529-551. Recuperado de: http://doi.org/10.1016/j.chb.2015.08.052. Gutiérrez-Braojos, C., Martín-Romera, A., Casasempere, A. y Fernández- Cano, A. (2015). Análisis cientimétrico de la Grounded Theory en Educación. Revista de Educación, 370, 121-148. Hanus, M.D. y Fox, J. (2015). Assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education, 80(1), 152-161. Recuperado de: http://doi.org/10.1016/j. compedu.2014.08.019. Heyvaert, M., Maes, B. y Onghena, P. (2013). Mixed methods research synthesis: definition, framework, and potential. Quality & Quantity, 47, 659–676. Jensen, L. y Allen, M. (1996). Meta-synthesis of qualitative findings. Qualitative Health Research, 6, 553–560. Kapp, K.M. (2012a). Games, gamification, and the quest for learner engagement. Training and Development, 66(6), 64-68.

128 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

Kapp, K. M. (2012b). The gamification of learning and instruction: Game- based methods and strategies for training and education. San Franciso, CA: Pfieffer. Koivisto, J. y Hamari (2014). Demographic differences in perceived benefits from gamification. Computers in Human Behavior, 35, 179- 188. Recuperado de: http://doi.org/10.1016/j.chb.2014.03.007. Lee, J. J. y Hammer, J. (2011). Gamification in education: what, how, why bother? Academic Exchange Quarterly, 15(2), 146. Lotka, A. (1926). The frequency distribution of scientific productivity. Journal of the Washington Academy of Sciences, 16(12), 317-323. Retrieved from http://www.jstor.org/stable/24529203. Malone, T.W. (1980). What makes things fun to learn? Heuristics for designing instructional computer games. In Proceedings of the 3rd ACM SIGSMALL symposium and the first SIGPC symposium on Small systems, pp. 162-169. Malone, T. W., y Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. Aptitude, learning, and instruction, 3, 223–253. Mangas-Vega, A., Gómez-Díaz, R., y Cordón-García, J. A. (2016). Approach to self-publishing with a combination of bibliometric study and social network analysis techniques. The electronic library, 34(6), 902-914. Martí-Parreño, J.; Seguí-Mas, D. y Seguí-Mas, E. (2016). Teachers’ Attitude towards and Actual Use of Gamification. Procedia - Social and Behavioral Sciences 228, 682 – 688. Mekler, E.D., Bruhlmann, F., Opwis, K., Tuch, A.N. (2013). Do points, levels and leaderboards harm intrinsic motivation? An empirical analysis of common gamification elements. In Proceedings of the First International Conference on Gameful Design, Research, and Applications, 66–73. Mekler, E.D., Bruhlmann, F., Tuch, A.N., Opwis, K. (2017). Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Computers in Human Behavior. 71, 525–534. Michán, L. y Muñoz-Velasco, I. (2013). Scientometrics for the medical sciences: Definitions, applications and perspectives. Revista de Investigación En Educación Médica, 2(6), 100-106. Recuperado de https://doi.org/10.1016/S2007-5057(13)72694-2.

Revista de Educación, 386. October-December 2019, pp. 109-131 129 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

Miguel, S. (2012) Ponencias o artículos: ¿una tensión en la comunicación científica?Anuario ThinkEPI, 6, 127-129. Ministerio de Educación y Formación Profesional (2018) Informe sobre el estado del sistema educativo del curso 2016/2017. Mollick, E. R. y Rothbard, N. (2014). Mandatory fun: Consent, gamification, and the impact of games at work, The Wharton School Research Paper Series. Recuperado de: http://dx.doi.ogr/10.2139/sssm.2277103. Palomo, J., Figueroa-Domecq, C. y Laguna, P. (2017). Women, peace and security state-of-art: a bibliometric analysis in social sciences based on SCOPUS database. Scientometrics, 113(1), 123-148. https://doi. org/10.1007/s11192-017-2484-x. Piaget, J. (1962). Play, dreams and imitation in childhood. New York: W.W. Norton & Co. Price, D. J. S. (1986). Little Science, big science…and beyond. New York: Columbia University Press. Price, D. (1971). The expansion of scientific knowledge. Annals Of The New York Academy Of Sciences, 184, 257-259. Powers, K.L., Brooks, P.J., Aldrich, N.J., Palladino, M.A. y Alfieri, L. (2013). Effects of video-game play on information processing: a meta-analytic investigation. Psychonomic Bulletin & Review, 20(6), 1055-1079. Recuperado de: http://doi.org/10.3758. Schacht, M., y& Schacht, S. (2012). Start the Game: Increasing User Experience of Enterprise Systems Following a Gamification Mechanism. En A. Maedche, A. Botzenhardt, y& L. Neer (Eds.), Software for People: Fundamentals, Trends and Best Practices (pp. 181–199). Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978- 3-642-31371-4_11. Seaborn, K. y Fels, D. (2015). Gamification in theory and action: a survey. International Journal of Human-Computer Studies, 74(1), 14-31. Recuperado de: http://doi.org/10.1016/j.ijhcs.2014.09.006. Simões, J., Díaz, R. y Fernández, A. (2013). A social gamification framework for a K-6 learning platform. Computers in Human Behavior, Elsevier, 29(2), 345–353. Su, C.H., y Cheng, C.H. (2015). A mobile gamification learning system for improving the learning motivation and achievements. Journal of Computer Assisted Learning, 31(3), 268–286. Thompson, D. F., y& Walker, C. K. (2015). A descriptive and historical review of bibliometrics with applications to medical sciences.

130 Revista de Educación, 386. October-December 2019, pp. 109-131 Received: 11-10-2018 Accepted: 17-05-2019 Parra-González, M.E., Segura-Robles, A. Scientific production on gamification in education: a scientometric analysis

Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy, 35(6), 551-559. Trujillo, F. (2014) Nuevos aprendizajes. Nuevos artefactos para nuevos aprendices. En F. Trujillo Sáez y Conecta13. Artefactos digitales: Una escuela digital para la educación de hoy. Barcelona: Graó. Wang, A. I. (2015). The wear out effect of a game-based student response system. Computers & Education, 82, 217-227. Recuperado de: http:// dx.doi.org/10.1016/j.compedu.2014.11.004. Whitton, N. y Moseley, A. (2010). Using Games to Enhance Learning and Teaching: A Beginner’s Guide. London: Taylor and Francis.

Contact address: Elena Parra González. Universidad de Granada, Facultad de Educación, Economía y Tecnología de Ceuta, Métodos de Investigación y Diagnóstico en Educación. C/. Cortadura del Valle s.n. CP: 51001 Ceuta. España. E-mail: [email protected]

Revista de Educación, 386. October-December 2019, pp. 109-131 131 Received: 11-10-2018 Accepted: 17-05-2019

Calculus tasks in Spanish University entrance exams

Tareas de Cálculo en las pruebas de acceso a la Universidad1

DOI: 10.4438/1988-592X-RE-2019-386-430

Juan Francisco Ruiz-Hidalgo Mª Elena Herrera Beltrán Mª Victoria Velasco Universidad de Granada

Abstract Entrance exams for admission to the university are generally considered as important in today’s society. Although various studies have been performed on the topic, this paper investigates in depth the Calculus problems posed in these exams in all autonomous communities in Spain in 2016. To perform the study, we use the method of content analysis, establishing and then duly adapting some task variables based on didactic analysis to organize the results. We then categorized these variables as descriptive, meaning-related, and cognitive, and interpreted the results according to this system of variables and categories. We observe that the meaning variable “mathematical content” and the cognitive variable “goal” are the ones that differ most among the autonomous communities in the tests analyzed. We also observe that other variables, such as the situation in which tasks are presented, are evaluated very differently in each autonomous community. The task profile most commonly used is the task stated verbally- symbolically, whose goal is to apply a procedure to analyze the behavior of a function as presented in a mathematical situation. This study constitutes a preliminary approach to detailed analysis of EBAU tasks that should be expanded, given the importance of the results.

(1) This study forms part of the CODAME research project funded by the Ministry of Science and Technology in Spain, of the national I+D+I Plan (Reference EDU 2015-70565P) and with the help of the group FQM-193 of the 3rd Andalusian Research Plan (PAIDI), MTM2016-76327-C3-2-P “Álgebras de Evolución y estructuras no asociativas”, funded by the Ministry of Science and Technology in Spain and FQM199 “Análisis Funcional y Aplicaciones”, Andalusian Research Plan (PAIDI).

Revista de Educación, 386. October-December 2019, pp. 133-158 133 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

Keywords: task analysis; admission conditions; assessment; non-compulsory school education; Spanish national curriculum; mathematics.

Resumen Las pruebas de acceso a la universidad se consideran de interés general en la sociedad actual. Aunque hemos encontrado diferentes investigaciones al respecto, profundizamos en los ejercicios de cálculo infinitesimal propuestos en las pruebas de acceso a la Universidad de las distintas comunidades autónomas españolas en el año 2016. Para ello, se utiliza el método de análisis de contenido y se toman, y se van adaptando razonadamente, unas variables de tarea basadas en el análisis didáctico que permiten la organización de los resultados. Estas variables, a su vez se categorizan como descriptivas, de significado y cognitivas. Los resultados se interpretan mediante este sistema de variables y categorías. Se observa que la variable de significado contenido matemático y la variable cognitiva meta son las que más diferencian las pruebas analizadas entre comunidades autónomas; y que otras variables, como la de la situación en la que se presentan las tareas, se valora de manera muy diferente en cada comunidad autónoma. El perfil de tarea más repetido es la que se enuncia de forma verbal-simbólica, cuya meta es aplicar algún procedimiento que trate de analizar el comportamiento de una función, por lo que se presenta en una situación matemática. Consideramos el estudio un primer acercamiento al análisis detallado de las tareas de las EBAU que, dado el interés de sus resultados, puede ser ampliado.

Palabras clave: análisis de tareas; condiciones de admisión; evaluación; enseñanza postobligatoria; currículum español; matemáticas.

Introducción

The first debates on university entrance examinations began in Spain at the end of the 19th century (González, 2001). These exams currently have extraordinary socio-educational value, since “regulation of the transition from secondary to higher education is a critical nexus of connection between the education system and society […] The fact that one of these is prioritized has great significance with both academic and social repercussions” (Muñoz, 1995, p. 32). The exam has had different names throughout its history. With the 2010 reform, the designation PAU (Prueba de Acceso a la Universidad/

134 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

University Entrance Exam) was changed to PAEG (Pruebas de Acceso a Estudios de Grado/Undergraduate Degree Entrance Exams) with the change in names for university degrees from licenciaturas and diplomaturas to Grado. In 2017, the Evaluación de Bachillerato para el Acceso a la Universidad (EBAU/Evaluation of Noncompulsory Secondary Education for Entrance to the University) was adopted (Ministerio de Educación Cultura y Deporte, MECD, 2016). Currently, the goal of the EBAU is to evaluate objectively the student’s academic maturity, as well as the knowledge and capabilities acquired in Bachillerato (non-compulsory secondary education) and the student’s ability to succeed in official undergraduate degree programs. Adapted to the Bachillerato curriculum established by law (Ministerio de la Presidencia, 2008), the EBAU is offered twice a year, with one regular and one special (make-up) sitting. Educational administrations and public universities are responsible for organizing the EBAU and for guaranteeing its adaptation to the curriculum in Bachillerato, as well as for coordinating with the university and the schools that offer Bachillerato to organize and hold the exam (Ministerio de la Presidencia, 2008). According to Muñoz (1995), three variables always affect design of the EBAU: number of possible candidates, number of positions available, and social needs for degree-holders in the various professions. These variables are in turn related to the nation’s economic and cultural development and the population’s employment expectations, among other issues. We did not find many recent (21st-century) studies of university entrance exams, at least not studies that focus on Mathematics. We do, however, find increased interest in them at the end of the 20th century, which materialized in a 1997 special issue of the Revista de Educación (no. 314). In this special issue, Escudero (1997) reviews the studies published to date and classifies them into four broad groups: ■ Studies of the model of entry to the university, including studies related to the model, outline, and organization system of the selection and entry process; ■ Studies of the internal characteristics of the exam itself, particularly studies by Muñoz-Repiso; ■ Studies of validation of the exam, which usually focuses on the relationship of the exam results to secondary school students; ■ Other studies that focus on alternative ways to enter the university. Also in this special issue, Murillo (1997) provides more in-depth analysis of the exams through analysis of the Mathematics exercises (and

Revista de Educación, 386. October-December 2019, pp. 133-158 135 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

other material) included in them. Murillo’s study analyzed little material and is thus hardly exhaustive or in-depth, as it focuses on units of content. It stresses that almost all Mathematics I exams asked about “Integrals” and “Spatial analytic geometry” but hardly any ask about “Determinants” or “Vector spaces.” In Mathematics II, the most common contents were “Calculus of probabilities” and “Systems of Linear Equations.” More recent studies by Grau, Cuxart and Martí-Recober (2002) discuss the quality of exam correction and recommend improving the system for exam preparation and correction. Boal, Bueno, Leríz and Sein-Echaluce (2008) focus on the mathematics skills required for the student to complete the test. Finally, Ruiz de Gauna, Balsera, Etxeberria and Sarasua (2013) categorize the studies based on objective: on results in different universities, comparative analysis, internal coherence of the exam, concurrence and predictive capability, some content or specific issues, and parallel modes of entry to the university. These authors focus on the results for Mathematics at the University of the Basque Country (Universidad del País Vasco-Euskal Herriko Unibertsitatea) over 14 years. They conclude that the entrance examination conditions the teaching methodology (differently for each subject in Mathematics) but does not condition the goal of learning Mathematics, which is always granted more importance than studying for the exam. Some studies have been performed of mathematics tasks in the EBAU. Those by Caraballo, Rico and Lupiáñez (2011a, 2011b) analyze the tasks proposed in the initial diagnostic tests held in Spain to determine the exam’s degree of fit to the PISA model (Program for International Student Assessment) and to determine the work that the autonomous communities performed in designing and applying the tests. The variables that the authors used were context, level of complexity, and content. The results are interpreted in terms of functionality of school mathematics. The functional model, which frames the current curriculum, focuses on development of mathematics competence—that is, on responding to real- life situations by applying the concepts learned and the mathematics abilities developed (Rico and Lupiáñez, 2008). The work presented here is descriptive. It focuses on the EBAU and seeks to answer the problems of determining the mathematics knowledge, skills, and competences of the students who finish noncompulsory secondary education. It also seeks to determine whether the EBAU can be considered as a useful exam for this purpose. More specifically, the

136 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams goal is to describe the EBAU exams for Mathematics II as they relate to the contents of Infinitesimal Calculus by analyzing the tasks contained in the exam. A secondary goal is to detect similarities and differences among the exams proposed in the different autonomous communities. The study thus falls within the group of studies on the internal characteristics of the exam in itself identified in Escudero (1997) or the group that studies specific content (Ruiz de Gauna, Balsera, Etxeberria and Sarasua, 2013. To achieve this goal, we use content analysis to analyze Calculus tasks in Mathematics II in the EBAU for 2016, applying the didactic analysis methodology (Rico, Lupiáñez and Molina, 2013). The results show the characteristics of these tasks, and the similarities and differences among the different autonomous communities.

Method

The study consists of a descriptive, exploratory investigation. It attempts to describe the tasks and the elements that characterize them. We find no similar prior studies (Cohen, Manion, & Morrison, 2011). Data analysis is performed after the content analysis to reveal the internal structure of the texts by studying their semantic content (Rico & Fernández-Cano, 2013). The data summary was performed by adapting the system of task variables described in Moreno and Ramírez (2016), which is being used similarly by other authors in current studies (among others, Herrera, Velasco, & Ruiz-Hidalgo, 2017; Jiménez, 2017; Martinez- Luaces, Rico, Fernández-Plaza, & Ruiz-Hidalgo, 2018; Vargas, Fernández- Plaza, & Ruiz-Hidalgo, 2018).

Sample

The first step in the research process was to gather the Mathematics II university entrance exams from the various autonomous communities throughout Spain for a specific period of time, 2012-2016. For operational reasons given the volume of the variables analyzed, we subsequently restricted the number of exams to focus on Mathematics II for 2016 in all autonomous communities in Spain: a total of 17 (exams from the two autonomous cities, Ceuta and Melilla, depend on the University

Revista de Educación, 386. October-December 2019, pp. 133-158 137 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

of Granada and were thus included under Andalusia). The study also focuses exclusively on task analysis of content related to content unit 3, Analysis (MECD, 2014b). Each task to be analyzed was assigned a 5-character code that includes two digits indicating the community to which the task belongs, whether the task belongs to the regular (R) or to the special (S) sitting in that community, number of the task, and test option to which it belongs (A or B).

Units of information

The exams are composed of various questions or exercises, of varying number depending on autonomous community. Since the numbers vary, we must present the results in terms of percentages. Further, some exercises are composed of various sections, which can be analyzed independently. To identify clearly the units of information considered –which we call tasks– we have taken the following considerations into account: ■ If the exercise is composed of independent sections, the different sections will be considered as independent tasks. In this case, these tasks share the same root code but specify the section, whether or not there is a relationship between them, point-value, formulation, etc. We thus hereafter use the term “task” to refer to a section that forms its own independent unit in the exercises. ■ We also found it helpful to make an exception to the previous point. Different sections of the same exercise are related when they refer to the same function or when one section asks the student to state a theoretical result and the other section to apply it. In other cases, the sections are considered as unrelated and independent tasks, even if they are based on the same content or goal.

Data analysis procedure

Once we had chosen the sample, we established a set of variables to organize and interpret the data. The variables were chosen using the reference framework of didactic analysis (Rico, Lupiáñez, & Molina,

138 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

2013; Rico & Moreno, 2016). This framework is divided into four types of analysis (content analysis, cognitive analysis, instructional analysis, and evaluation analysis) based on the dimensions of the mathematics curriculum (cultural/conceptual, cognitive, ethical/formative and social). In didactic analysis, analysis of school mathematics tasks is one of the organizers used to analyze instruction, planning, and implementation, and one part of this analysis is study and design of tasks. To perform this analysis, we investigate the variables in the task, functions of the tasks, and different types of task present (Moreno & Ramírez, 2016). Initial choice of the variables was based on the criteria of Moreno and Ramírez (2016). These variables enable inclusion of the aspects deemed most relevant when analyzing a task and extracting nearly all information from it. The processes of data collection and organization required elimination of some variables and more concrete specification of others. For example, “Materials” was initially considered as a variable that gathered information on the materials needed to complete the task. This variable was eliminated because it contributed no difference among the tasks. Next, we describe the variables ultimately used to describe the exams and the tasks: ■ Language in which the task is expressed. ■ Formulation. Describes the system of representation in which the task is expressed. ■ Goal. Refers to the task’s intention. Expresses the capability to be put into play to resolve the task. ■ Content. Determines in the most specific way possible in Calculus the content to which the task refers. The categories that emerged include Continuity, Calculation of limits, Asymptotes, among others. ■ Complexity. Enables us to describe the difficulty of the task. Again following Moreno and Ramírez (2016), and based on the PISA studies (OECD, 2009), we consider three degrees of difficulty: Reproduction, Connection, and Reflection. We were unable to classify the tasks on contents of integration, calculation of areas, and calculation of limits using these criteria, since complexity varies based on difficulties that might arise during application of the procedures or due to the type of function. ■ Situation. Refers to the context in which the task is proposed. To make data collection more efficient, we considered the situations

Revista de Educación, 386. October-December 2019, pp. 133-158 139 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

based on the PISA framework (OECD, 2009). These situations are personal, educational or occupational, public, and scientific. Within “scientific,” we introduced a value for specificity that distinguishes the nature of the branch of science to which the task belongs (scientific or technological). ■ Number of sections in the exercise (if it contains more than one). ■ Point-value (of each of the sections, where there is more than one). The data were gathered and coded, but since not all autonomous communities included them, we could not analyze this variable. ■ Relationship among the sections. Considers the different sections of the same task as related when they refer to the same function or when one section requires stating a theoretical result and the other applying it. In addition to these variables taken from the reference texts, the process of analysis required addition of the following variables in order to include all information provided by the tasks: ■ Type of function. Enables one to determine more precisely aspects of the mathematics content to be manipulated and difficulty of the procedures. Distinguishes between: A) elementary functions, trigonometric functions, and scalar multiples of these functions; and B) functions defined as composites of elementary functions, as a linear combination of elementary functions, and as functions defined using parameters. ■ Type of exercise. Distinguishing between: A) Direct: statement requires performance of a series of actions and procedures to obtain a series of results (we do not know the results beforehand); B) Inverse: statement indicates what should be accomplished; we are generally asked to perform an inverse procedure, whether to find the function or to calculate a series of parameters that enable us to accomplish what the exercise asks. Following the criteria in Vargas, Fernández-Plaza and Ruiz-Hidalgo (2018), it is logical to organize the variables according to aspects of the reference framework. To do so, we use three sets of variables: First, the language, number of sections, point-value, and relationship among sections are classified into the set of descriptive variables for the exam. Second, we consider goal, complexity, and type of task as variables of the tasks related to cognitive issues. Finally, a third group

140 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams of variables on tasks provides information on their content and its meaning (Rico, 2012, 2013; Rico & Moreno, 2016; Fernández-Plaza et al., 2016). These are content, type of function, formulation, and situation. In school mathematics, the meaning of a concept is understood as “knowing its definition; how to represent it; how to demonstrate its operations, relationship and properties; and its categories of use, as well as interpretation and application to solving problems” (Rico, 2016, p. 94).

Results

The results are organized by the groups of variables described: descriptive variables of the test, variables related to cognitive issues, and finally variables related to the meaning of the mathematical concept to which they refer.

General aspects of the exams

The exam is not offered in Castilian Spanish in the autonomous communities of Catalonia, Galicia, and the Balearic Islands. Autonomous communities such as the Basque Country and Valencia (as well as the other communities not mentioned) do offer the exam in Castilian Spanish. As to choice of tasks on the exam, all communities except Catalonia permit choosing from one of two options (Option A and Option B) on all exams. The student must choose to do one of the two and perform all tasks included in that option. In no case—and this is true for all autonomous communities—may the student complete exercises in both options. In the regular sitting of the exam in Catalonia, as in the other autonomous communities, two options are provided, but in the special sitting Catalonia provides only one. In both cases, the options include six tasks, of which the student must complete five. Another general issue is the point-values of the tasks. Most of the autonomous communities specify the point-value of each exercise on the entrance exam, and the overall grade is obtained by adding all partial point-values, which generally total 10 points. Numerous differences exist among the different communities on this issue, however. In the Balearic Islands and Valencia, for example, each exercise is worth 10 points. In

Revista de Educación, 386. October-December 2019, pp. 133-158 141 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

Extremadura, the test explains at the beginning that each task is worth a maximum value of 2.5 points. Exams in the Basque Country indicate that each exercise is worth a maximum of 2 points. Finally, the communities of La Rioja and the Canary Islands do not assign any point values. The first striking difference among the exams in the different autonomous communities is the number of tasks proposed (more exhaustive study would require analyzing each section of each exercise). For example, Cantabria is one of the autonomous communities with the “lightest” number of exercises per exam, while Asturias’ exams are the most “heavily” loaded with exercises. Oddly, communities such as La Rioja divided their exercises into many sections. We chose to analyze the results in terms of percentages to solve this problem of disparity.

Aspects of meaning of the derivative

This section includes the variables for task formulation, content, type of function, and situation.

Formulation

At national scale, verbal-symbolic language predominates primarily in the statements of the tasks; it is used in 92.1% of all autonomous communities. Exclusively verbal language follows, at great distance –6.7% (used only in Asturias, the Balearic Islands, Cantabria, Castilla-La Mancha, Castile and León, Extremadura, and Galicia). More specifically, verbal language is used extensively in Cantabria, where 40% of tasks are stated in words only, and in Galicia (20%). Finally, verbal-graphic language is used least. In fact, it is only used in Aragón, Asturias, and the Canary Islands (overall percentage 1.2%), with the Canary Islands using it most (7.7%). Figure I presents this information by autonomous community.

Content

The variable of content associates the task with the mathematical goal

142 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams to be achieved. Many authors use this variable in their research, but since they usually use it in school mathematics contexts (primary and secondary), the variable is usually classified according to school content units (MECD, 2014a, 2014b). Since the school classification does not reflect the richness of the content in the EBAUs, we define the categories associated with the variable based on the specific contents. Further, we add a category Non-classifiable (NC) for sections that do not refer to a specific content.

FIGURE I. Representation in formulation of tasks

Source: Developed by the authors

Due to the large number of categories that have emerged for this variable content (47), we reclassified the categories to make them more convenient for operational purposes, to reduce the size of the table displaying the results. In some cases, we grouped several related categories under an existing heading. For example, the initial categories “derivation” (calculating the derivative of a function) and “derivability” (studying whether a function is differentiable) were regrouped under the heading “derivability.” In other cases, we created a new category. For example, “theorems” encompasses some of the categories established at the beginning (such as Rolle’s Theorem or the Fundamental Theorem of

Revista de Educación, 386. October-December 2019, pp. 133-158 143 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

Calculus, among others). With this regrouping, we manage to describe the variable “content” proportionately using the 10 following categories: “Behavior of a function,” which encompasses all categories related to sketching a function and its basic descriptors; “Continuity”, “Derivability”, “Optimization”, “Integration”, “Calculation of areas”, “Theorems” (which includes all theoretical results), “Modeling function”, “Calculation of limits, and “NC.” It is useful to know that data collection was “finer” for the categories ultimately adopted, as the data can be used in future studies, with other parameters. This process in turn grounds the final reclassification adopted.

TABLE I. Content in statements of tasks

Content Percentage Behavior of a function 32.41% Integration 16.60% Calculation of areas 13.44% Derivability 12.25% Calculation of limits 9.49% Theorems 5.93% Optimization 4.74% Continuity 2.77% Modeling function 1.58% NC 0.79%

Source: Developed by the authors

Based on the data in Table I, the most common content is related to behavior of a function (32.41%), followed by integration (16.60%), and calculation of areas (13.44%) (totaling one third of the content). In fourth and fifth place, we find derivability (12.25%) and calculation of limits (9.49%). The contents that might be the most “problematic” for students— alluding to tasks related to theorems and problems of optimization— represent around 10% of the total. The content on “behavior of a function and how to draw it” is the most common in the national territory and is present in all autonomous

144 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams communities in very high percentages. Valencia stands out, with 63.16% content, followed by Cantabria (60%) and the Balearic Islands (57.14%). Andalusia dedicated nearly half (46.15%) of its tasks to studying behavior of a function, in contrast with autonomous communities like Navarre and Castile and León, which dedicate less than or equal to 10% of their tasks to this activity. Content on integration is present in all communities except Cantabria and Castile and León, a result we find striking. However, this content constitutes over 40% in the Murcia and over 30% in the Community of Madrid. Further, it constitutes 16.60% of the national total, quite a high percentage and the second-most-frequent content. The category “calculation of areas” constitutes the third-most-common content at national level. It is present in all autonomous communities with a content of over 5.5% (in contrast, to calculation of an antiderivative), constituting over 30% in Castile and León and 20% in Navarre. The communities least interested in these contents are Castilla-La Mancha (6.67%), Extremadura (5.56%), and Murcia (5.88%). Oddly, Murcia is the community that requires the most calculation of antiderivatives in its university entrance exams. Table II shows the most commonly used content, by autonomous community.

TABLE II. Content most commonly used, by autonomous community

Community Most common content Andalusia Behavior of a function, 46% Aragón Behavior of a function, 29% Asturias Behavior of a function, 29% Balearic Islands Behavior of a function, 57% Canary Islands Behavior of a function, 38% Cantabria Behavior of a function, 60% Castile and León Calculation of areas, 30% Castilla-La Mancha Behavior of a function, 27% Catalonia Derivability, 25% - Behavior of a function, 25% Valencia Behavior of a function, 63% Extremadura Behavior of a function, 33%

Revista de Educación, 386. October-December 2019, pp. 133-158 145 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

Galicia Derivability, 33% Madrid Integration, 33% Murcia Integration, 41% Navarre Theorems, 30% Basque Country Behavior of a function, 31% La Rioja Behavior of a function, 29%

Source: Developed by the authors

Type of function

Type of function is the variable that specifies the kind of function the student is to manipulate in a task. Our analysis establishes two categories for this variable: type A and type B functions. The latter determine, to some degree, the category of the following (type of task), since the tasks that include functions with parameters are classified as Inverse-type exercises. As with the previous variable, we add a third mode, NS (not specified) for cases in which the function is not determined in the task, as could be the case, for example, in an optimization exercise. In the national territory, the percentage of tasks corresponding to function types A and B is nearly the same (43.87% Type A and 45.45% Type B). We can also affirm that most of the tasks proposed establish the type of function required; the exercises in which the function must be determined represent only 10.67% of cases. The number of tasks using type-A and type-B functions may differ within the same autonomous community. In fact, the difference is especially striking in the Basque Country, where type-A functions represent over 70%, and type B only slightly over 10%. In Murcia and Navarre, in contrast, type-B functions are very close to 70% but type-A less than 30%. The same occurs in Castilla-La Mancha, where type-B functions are close to 60%, whereas type-A functions are around 25%. In Cantabria, on the other hand, the highest percentage of this variable is represented by non-specified functions, in contrast to the overall totals. In communities such as Catalonia and Galicia, it is also noteworthy that non-specified functions represent over 20%, whereas other communities –such as the Community of Valencia, Madrid, Murcia and Navarre– all specify the type of function to be manipulated.

146 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

Table III shows the most common type of function in tasks by community. We use the combination “Type A Function-Type B Function” in Andalusia because the two types occur there in equal proportion.

TABLE III. Type of function most commonly used, by autonomous community

Community Most common content Andalusia Type A- Type B Aragón Type B Asturias Type A Balearic Islands Type B Canary Islands Type A Cantabria Not specified Castile and León Type A Castilla-La Mancha Type B Catalonia Type B Valencia Type B Extremadura Type A Galicia Type B Madrid Type B Murcia Type B Navarre Type B Basque Country Type A La Rioja Type A

Source: Developed by the authors

Situation

The situation is the variable that provides the sense of utility to the tasks and shows different ways of using the mathematical concepts. After reviewing the different tasks proposed in the EBAU, we established three categories for this variable: Mathematics, for purely mathematical situations; Scientific, for scientific situations that were not mathematical; and Personal/Occupational/Public, for what could be called real-life situations, which include Work-related, Personal, and Public situations. Table IV presents the results in the national context.

Revista de Educación, 386. October-December 2019, pp. 133-158 147 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

TABLE IV. Situation in the task statements

Situation Percentage Mathematics 90.91% Personal/Occupational/Public 5.94% Scientific 3.17%

Source: Developed by the authors

The results do not permit us to establish too many differences among the tasks evaluated. We find a high level of tasks stated in strictly mathematical situations (90.91%). This result shows that tasks proposed in University Entrance Exams are generally not designed in real contexts, or show almost no connection of their statements to real-life situations. Further, only two autonomous communities include tasks in a (non- mathematic) scientific context: Valencia (26.32%) and Murcia (17.65%). Valencia is the only community in which the three situations analyzed are handled, and even there, exclusively mathematical tasks represent 57.89% of the exam. The case of Cantabria is noteworthy for its high percentage of tasks in the Personal/Occupational/Public category (40%). Its exams do not, however, use scientific contexts. Catalonia is similar but shows a percentage of 16.67% for this category. Other autonomous communities that include problems in the Personal/Occupational/Public context, although at percentages under 10%, are Aragón, Asturias, the Canary Islands, Castile and León, and Castilla-La Mancha.

Cognitive aspects of the tasks

Cognitive aspects include the variables goal, complexity, and type of exercise.

Goal variable

The goal variable refers to the expectation or expectations to which the task is oriented. For this case, we establish the following: Apply

148 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams a procedure, Reasoning and argument, Represent graphically, State a theoretical result (which includes State a theorem, State a definition, and State a rule), Apply a theoretical result (which includes Apply a theorem, Apply a definition, and Perform a proof by applying one of the theorems), and Modelling, as well as combinations of these goals. Although it is true that all modes of this variable are present in the tasks analyzed, the results generally show that the tasks composing the university entrance exams attempt to determine that the student is able to apply procedures. Table V shows the percentages obtained at national level in our analysis of this variable.

TABLE V. Goal of statements in the tasks

Goal Percentage Apply a procedure 73.91% Reasoning and argument 9.88% Represent graphically 8.70% Apply a theoretical result 4.35% State a theoretical result 2.77% Modelling 0.4%

Source: Developed by the authors

As mentioned, the most common goal on exams is to apply a procedure, which constitutes 73.91% of the tasks. Reasoning and argument and representing graphically follow, but far behind (9.88% and 8.70%, respectively). Graphic representation is quite highly valued, and a great number of tasks require not only applying a procedure but also reasoning and making an argument based on the results obtained from these procedures. We thus find a series of tasks that cannot be classified as “mechanical,” as they also require connecting and relating different representations or connecting different aspects to find the solution. This is the case of graphic representation and reasoning using the results obtained by justifying and arguing their meaning in the case of applying procedures.

Revista de Educación, 386. October-December 2019, pp. 133-158 149 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

It is also noteworthy that we obtain relatively low data on theoretical results and their application. Only 2.77% of the tasks require stating a theoretical result and only 4.35% applying it. In Table VI, we identify the second-most-common goal in each community. (Application of a procedure is the most common goal in all communities). Note that the autonomous communities of Aragón, Catalonia, Asturias and Extremadura require special consideration, since they have various secondary categories in equal percentages, preventing us from considering them as like the others. The Community of Madrid, in turn, cannot be classified like the others, since 100% of its tasks require application of a procedure.

TABLE VI. Second-most-common goal of the tasks, by autonomous community

Community Second-most-common goal Andalusia Represent graphically Aragón Various Asturias Various Balearic Islands Reasoning and argument Canary Islands Represent graphically Cantabria Represent graphically Castile and León Reasoning and argument Castilla-La Mancha Apply a theoretical result Catalonia Various Valencia Reasoning and argument Extremadura Various Galicia State a theoretical result Madrid Apply a procedure Murcia Reasoning and argument Navarre Apply a theoretical result Basque Country Reasoning and argument La Rioja Represent graphically

Source: Developed by the authors

150 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

Complexity

The complexity of the task enables us to describe its difficulty. The categories considered were reproduction, connection, and reflection. We also added another category for this variable, NC (non-classifiable) for the cases in which one cannot classify the task a priori based on its difficulty. Such cases arise when, for example, the task requires solving an integral, calculating a limit, or finding an area, tasks in which difficulty is directly related to the procedure. This is reflected to a great extent in the value of the other variable in the analysis, type of function. That the categories are clearly insufficient to classify all of the tasks we have marked as non-classifiable (NC), and that these tasks represent quite a high percentage of the tasks analyzed, suggest that the instruments for classifying complexity are insufficient for studying certain tasks because classification of these tasks can vary based on how one tackles resolving the tasks (For example, in calculating an antiderivative, complexity may clearly depend on the method of integration used.) The reason for this problem may be that these categories were originally designed to classify school tasks, whose difficulty is more predictable and easier to estimate. For the variable complexity, as for the other variables analyzed above, Figure II presents the data collected by autonomous community relative to the general total. The most common categories of the variable complexity are NC (non- classifiable) and tasks on reproduction. Tasks on reflection appear less frequently. We see that the entrance exams analyzed from Andalusia, Cantabria, Valencia, Galicia, Madrid, Murcia, and the Basque Country contain no tasks on reflection. The communities of Navarre and Castilla- La Mancha use reflection tasks more often. Further, in Castilla-La Mancha, the number of tasks on reflection and NC tasks is similar, although tasks on reproduction predominate.

Revista de Educación, 386. October-December 2019, pp. 133-158 151 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

FIGURE II. Most common level of complexity, by autonomous community

Source: Developed by the authors

The tasks on reproduction are especially important in Catalonia and Valencia (with percentages close to 60%), followed by Andalusia and the Balearic Islands. The autonomous communities that use these tasks least are Castile and León, Asturias, Navarre, and the Basque Country (percentages below 10%). Tasks on connection predominate in Cantabria and are also present, although to a lesser extent, in the Basque Country, Madrid, the Canary Islands, La Rioja, Asturias, and Extremadura. Exams in the Canary Islands show similar proportions of tasks on reproduction, connection, and NC. Table VII identifies the most common complexity by community.

TABLE VII. Complexity of most common tasks, by autonomous community

Community Complexity Andalusia Reproduction Aragón NC Asturias NC Balearic Islands Reproduction Canary Islands Reproduction-Connection-NC

152 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

Cantabria Connection Castile and León NC Castilla-La Mancha Reproduction Catalonia Reproduction Valencia Reproduction Extremadura Reproduction Galicia Reproduction-NC Madrid Reproduction Murcia NC Navarre NC Basque Country Connection-NC La Rioja NC

Source: Developed by the authors

Type of exercise

The variable type of exercise has two modes in our study: Direct and Inverse. With the help of Figure III, we present the results obtained by autonomous community. To contrast them in perspective, we included the data on the national average in the last bar on the right, labelled TOTAL. In view of the data in Figure III, direct tasks predominate nationally (83.79%) and outweigh inverse tasks in all autonomous communities (direct tasks occur in at least 73% of the tasks). There are even communities that propose only direct tasks, such as the Balearic Islands and Navarre, whose values contrast with the national reference value. Inverse tasks are above 20% in Castilla-La Mancha, Murcia, the Basque Country, Asturias, and Extremadura.

Revista de Educación, 386. October-December 2019, pp. 133-158 153 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

FIGURE III. Type of task, by autonomous community

Source: Developed by the authors

Conclusions

Throughout this study, we have presented a large amount of information and sought to interpret it as clearly and objectively as possible, using different forms of organization and presentation of the results to synthesize the different categories that served as variables for our analysis. As to the goal of describing the EBAU exams for Mathematics II, we have performed an exhaustive description of the tasks for Calculus that compose these exams, and the variables content and goal have been key in identifying the tasks. The differences among autonomous communities are shown in the three groups of variables: descriptive variables of the exams, variables on cognitive issues, and variables on meaning. Of the group of cognitive variables, we stress the variable goal, where the capability of applying a procedure is the category most frequently repeated throughout Spain (around 73% of tasks), to the detriment of more complex mathematics skills such as reasoning, representing, stating, etc. Among these variables of meaning, only some autonomous

154 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams communities are interested in questions that go beyond study of the behavior of functions, and the situations are always mathematical except in Valencia and Cantabria. These issues show that—at least as regards Calculus—the EBAU focuses little on evaluation of students’ mathematical competence. On the one hand, the demands made in the tests are primarily basic skills of immediate application and do not include more complex procedures related to skills that usually describe competence (OECD, 2009). The absence of situations external to mathematics is a second clear symptom of lack of interest in evaluating development of mathematical competence and in the functional focus of school mathematics, since there is no opportunity for application. Despite this contradiction between the goal of developing mathematical competence in Bachillerato and lack of attention to mathematical competence in the EBAU itself, authors such as Ruiz de Gauna, Dávila, Etxeberria and Sarasausa (2013) argue that the exam does not condition the goal of learning mathematics in secondary schools. In any case, the EBAU focuses on evaluating mathematics skills and the kind of tasks that indicate a propedeutic aim in Mathematics II, as previously indicated by Boal, Bueno, Lerís and Sein-Echualce (2008). One final conclusion is that the instrument of task analysis presented here has been effective. All of the variables provided interesting information except the variable complexity (originally taken from the PISA studies and thus oriented to classifying tasks in compulsory education), which is shown to be insufficiently defined to characterize the EBAU tasks. The information provided here can be important to Mathematics teachers in both secondary and university instruction, as it gives a carefully-argued, in-depth view of what the EBAUs require so that teachers can learn in their everyday actions to take the position they believe most appropriate –or at least have information on this issue. For pragmatic reasons, we limited the study sample to exams from the year 2016 and to only one subject (Mathematics II). Continuing this study but widening the scope of both of these aspects could be another natural way to further this research. Such an approach would enable longitudinal comparisons across both time and subject matter.

Revista de Educación, 386. October-December 2019, pp. 133-158 155 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

References

Boal, N., Bueno, C., Lerís, M. D., & Sein-Echualce, M. L. (2008). Las habilidades matemáticas evaluadas en las pruebas de acceso a la universidad. Un estudio en varias universidades públicas españolas. Revista de investigación educativa, 26 (1), pp. 11-23. Caraballo, R. M., Rico, L., & Lupiáñez, J. L. (2011a). Análisis de los ítems de las evaluaciones autonómicas de diagnóstico en España 2008-2009. Unión, 26, 27-40 Caraballo, R. M., Rico, L., & Lupiáñez, J. L. (2011b). Pruebas autonómicas de diagnóstico para evaluar la competencia matemática en educación secundaria. In M. Marín, G. Fernández, L. J. Blanco, & M. Palarea (Eds.), Investigación en Educación Matemáticas XV (pp. 307-318). Ciudad Real: SEIEM. Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education. London: Routledge. Escudero, T. (1997). Investigaciones sobre el proceso de selección de universitarios en España: Revisión comentada. Revista de Educación, 314, 7-27. Fernández-Plaza, J. A., Castro-Rodríguez, E., Estrella, M., Martín- Fernández, E., Rico, L., Ruiz-Hidalgo, J. F., & Vílchez-Marín, M. (2016). Significado y concepciones de conceptos matemáticos escolares. En C. Fernández, J. L. González, F. J. Ruiz, T. Fernández y A. Berciano (Eds.), Investigación en Educación Matemática XX (pp. 259-268). Málaga, Spain: SEIEM. González, A. (2001). Los orígenes de la selectividad en la Universidad española: el examen de ingreso en facultades (1898-1902). Hispania, 61 (207), pp. 315-337. Grau, R., Cuxart, A., & Martí-Recober, M. (2002). La calidad en el proceso de corrección de las pruebas de acceso a la universidad: variabilidad y factores. Revista de investigación educativa, 20 (1), pp. 209-223. Herrera, M. E., Velasco, M. V., & Ruiz-Hidalgo, J. F. (2017). Comparando textos de cálculo: el caso de la derivada. PNA, 11(4), 280–306. Jiménez, A. (2017). Significados de la derivada en las pruebas de evaluación de bachillerato para el acceso a la universidad (unpublished Master’s thesis). University of Granada, Granada, Spain. Martinez-Luaces, V., Rico, L., Fernández-Plaza, J.A., & Ruiz-Hidalgo, J.F. (2018). Enriquecimiento de tareas y problemas de modelado inverso:

156 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

Una experiencia con profesores en formación. In L. J. Rodríguez- Muñiz, L. Muñiz-Rodríguez, A. Aguilar-González, P. Alonso, F. J. García García, & A. Bruno (Eds.), Investigación en Educación Matemática XXII (pp. 320-329). Gijón: SEIEM. Ministerio de Educación, Cultura y Deporte, MECD (2014a). Real Decreto 126/2014, de 28 de febrero, por el que se establece el currículo básico de la Educación Primaria. Ministerio de Educación, Cultura y Deporte, MECD (2014b). Real Decreto 1105/2014, de 26 de diciembre, por el que se establece el currículo básico de la Educación Secundaria Obligatoria y del Bachillerato. Ministerio de Educación, Cultura y Deporte, MECD (2016). Orden ECD/1941/2016, de 22 de diciembre, por la que se determinan las características, el diseño y el contenido de la evaluación de Bachillerato para el acceso a la Universidad, las fechas máximas de realización y de resolución de los procedimientos de revisión de las calificaciones obtenidas, para el curso 2016/2017. Ministerio de la Presidencia (2008). Real Decreto 1892/2008, de 14 de noviembre, por el que se regulan las condiciones para el acceso a las enseñanzas universitarias oficiales de grado y los procedimientos de admisión a las universidades públicas españolas. Murillo, F. J. (1997). Análisis de las pruebas que conforman la selectividad. Revista de Educación, 314, 49-62. Moreno, A., & Ramírez, R. (2016). Variables y funciones de las tareas matemáticas. En L. Rico y A. Moreno (Coords.), Elementos de didáctica de la matemática para el profesor de secundaria (pp. 243-257). Madrid, Spain: Pirámide. Muñoz, F. (1995). El acceso a la Universidad en España: perspectiva histórica (1). Revista de Educación, 308, pp. 31-61. OECD (2015). PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematics and Financial Literacy. Paris: PISA, OECD Publishing. Rico, L. (2012). Aproximación a la investigación en Didáctica de la Matemática. Avances de Investigación en Educación Matemática, 1, 39–63. Rico, L. (2013). El método del Análisis Didáctico. Revista Iberoamericana de Educación Matemática, 33, 11–27. Rico, L., & Fernández-Cano, A. (2013). Análisis didáctico y metodología de investigación. In L. Rico, J. L. Lupiañez, & M. Molina, (Eds.)

Revista de Educación, 386. October-December 2019, pp. 133-158 157 Received: 28-12-2018 Accepted: 17-05-2019 Ruiz-Hidalgo, J.F., Herrera Beltrán, M.E., Velasco, M.V. Calculus tasks in Spanish University entrance exams

(2013). Análisis Didáctico en Educación Matemática: metodología de investigación, formación de profesores e innovación curricular. Granada: Editorial Comares. Rico, L., & Lupiáñez, J. L. (2008). Competencias matemáticas desde una perspectiva curricular. Madrid: Alizanza Editoral. Rico, L., Lupiañez, J. L., & Molina, M. (Eds.) (2013). Análisis Didáctico en Educación Matemática: metodología de investigación, formación de profesores e innovación curricular. Granada: Editorial Comares. Rico, L., & Moreno, A. (Eds.). (2016). Elementos de didáctica de la matemática para el profesor de Secundaria. Madrid, Spain: Ediciones Pirámide. Ruiz de Gauna, J., Dávila, P., Etxeberria, J., & Sarasausa, J. M. (2013). Pruebas de selectividad en Matemáticas en la UPV-EHU. Revista de Educación, 362, 217-246. Vargas, M. F., Fernández-Plaza, J. A., & Ruiz-Hidalgo, J. F. (2018). Tareas propuestas por los libros de texto de 1º de bachillerato para el tema de derivada. In L. J. Rodríguez-Muñiz, L. Muñiz-Rodríguez, A. Aguilar- González, P. Alonso, F. J. García García, & A. Bruno (Eds.), Investigación en Educación Matemática XXII (pp. 594-603). Gijón: SEIEM.

Contact address: Juan Francisco Ruiz Hidalgo. Universidad de Granada, Facultad de Ciencias de la Educación, Departamento de Didáctica de la Matemática. Facultad de Ciencias de la Educación. Campus de Cartuja, s/n. Universidad de Granada. CP 18071. Granada. E-mail: [email protected]

158 Revista de Educación, 386. October-December 2019, pp. 133-158 Received: 28-12-2018 Accepted: 17-05-2019 The impact of relative age on results in language and mathematics in primary education in Asturias, Spain1

Impacto de la edad relativa en los resultados en lengua y matemáticas en educación primaria en Asturias

DOI: 10.4438/1988-592X-RE-2019-386-431

Paula González-Vallinas E.O.E. Avilés Julián Librero Navarrabiomed, Complejo Hospitalario de Navarra (CHN)-Universidad Pública de Navarra (UPNA), IDISNA Salvador Peiró Fundación para el Fomento de la Investigación Sanitaria y Biomédica de la Comunidad Valenciana (FISABIO) José Luis San Fabián Universidad de Oviedo

Abstract Various studies have examined the influence of a variety of student-related variables on student academic performance and results in schools and in regional and national achievement tests. However, relative age is still a missing variable in many of these studies, yet to address the fact that the same tests are applied to all students in a class, with some of them being as much as almost a year younger than their classmates. This paper explores the impact of birth month on achievement in language and mathematics in grades 2, 4 and 6 in primary school pupils in the Principado de Asturias, Spain, during academic years 2008- 2013. First, bivariate analyses were performed to assess the association between final course marks and student birth month, to determine the percentage of year repeats by birth month, and finally, the relationship between educational support

(1) We would like to express our gratitude to the Department of Education and Culture of the Principado de Asturias (Spain) for providing the data.

Revista de Educación, 386. October-December 2019, pp. 159-180 159 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

requirements and relative age. Following that, a multilevel logistic regression analysis was performed to determine whether the likelihood of achieving an outstanding score in mathematics or language differed for students born in the final quarter of the year, controlling for the effect of sex, school year and school attended. The results show that students born in the last few months of the year were more likely to have lower marks, repeat years and receive educational reinforcement in language and mathematics, as well as being less likely to achieve outstanding marks. These results suggest that relative age may be a significant variable when evaluating school results.

Keywords: Relative age; birth month; academic achievement; primary education; educational assessment.

Resumen Numerosos estudios han analizado el efecto de diversas variables del alumnado sobre su rendimiento escolar, así como sobre los resultados en las pruebas de diagnóstico de las diferentes Comunidades Autónomas. Sin embargo, el mes de nacimiento todavía no se contempla como variable en muchos de los estudios, obviando el hecho de que en las aulas se aplican las mismas pruebas a niños y niñas donde la diferencia de edad es a veces de hasta casi un año, sin que sus notas sean ajustadas en función de su edad relativa. Esta investigación analiza el impacto del mes de nacimiento del alumnado en las notas en Lengua y Matemáticas de los cursos de 2º, 4º y 6º de Educación Primaria en el Principado de Asturias (España) en el periodo 2008-2013. En primer lugar se realizó un análisis bivariable para valorar la asociación entre notas finales del respectivo curso y el mes de nacimiento, así como para determinar el porcentaje de repetición de curso según mes y, finalmente, la relación entre el mes de nacimiento y tener refuerzo educativo. A continuación se realizó un análisis de regresión logística multinivel para contrastar si la propensión a obtener sobresaliente en matemáticas o en lengua difiere para el alumnado nacido en el último trimestre del año controlando el efecto del sexo y del curso y el nivel colegio al que acuden. Los resultados muestran que el alumnado nacido en los últimos meses del año tiene una mayor probabilidad de obtener peores notas, repetir curso y recibir refuerzo educativo, y una menor probabilidad de obtener la calificación sobresaliente en ambas asignaturas. Los resultados sugieren que la edad relativa debería ser una variable relevante al analizar los resultados escolares,

Palabras-clave: Edad relativa; mes de nacimiento; resultados escolares; educación primaria; evaluación educativa.

160 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

Introduction

The first international research analyzing the impact of student relative age on school results appeared more than thirty years ago. Relative age is the difference in ages between students in the same class (Kinard & Reinherz, 1986; Allen & Barnsley, 1993), which can be up to a year for those born at the beginning of January and those born at the end of December. Schooling in compulsory education in many European countries, including Spain, is organized according to the year of birth, or the age when school starts such as in the United States (where older children are born in September, and the youngest are born in the summer). This means that there can be up to almost a year’s age difference between the oldest and youngest in the same class, a difference that could be significant from an educational point of view for children aged between 6 and 12 (Kawaguchi, 2006; Puhani & Weber, 2007; Black, Devereux & Salvanes, 2008; Crawford, Dearden & Greaves, 2013). Although there is growing evidence of the influence of relative age on school achievement (Strom, 2004; Bedard & Dhuey, 2006; La Paro & Pianta, 2000; Oshima & Domaleski, 2006; Sharp, George, Sargent, O’Donnell, & Heron, 2009; Sprietsma, 2010; Robertson, 2011), birth month is still a variable which is not considered when organizing class groups or analyzing school results, whether within the educational system or in international tests, where students take the same test on a single date regardless of their relative age at that moment (Sampaio, da Matta, Ribas, & Sampaio, 2016; Smith, 2010; Martin, Martínez, Marchesi & Pérez, 2008). In Spain, annual diagnostic tests in primary education are performed in various autonomous communities, with Cantabria being the only one where birth month appears as a variable related to the results in the annual diagnostic test reports (Consejería de Educación, Cultura y deporte. Gobierno de Cantabria, 2013). Various studies have demonstrated that on average, students born towards the end of the year have worse results, not because they are the youngest in absolute terms, but because they are the youngest in relation to their class group (Daniels, Shorrocks-Taylor & Redfern, 2000; Puhani & Weber, 2005; Lawlor, Clark, Ronalds, & Leon, 2006; Navarro, García- Rubio & Olivares, 2015): the youngest students in each class suffer the inequality of being evaluated according to their abilities and knowledge in homogeneous tests that do not consider relative age, and thus are

Revista de Educación, 386. October-December 2019, pp. 159-180 161 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

at a clear disadvantage compared to their older classmates (Sprietsma, 2010; Thoren, Heinig & Brunner, 2016). Research also indicates that there is a positive, linear relationship between relative age in a school year and school marks, older students in a class get better results than the younger students, and while these differences tend to diminish as schooling continues, they still exist and end up affecting future study, beyond compulsory education, even having an impact on the likelihood of attending university (Boardman, 2006; Sykes, Bell & Rodeiro, 2009; Crawford, Dearden & Greaves, 2011; Bedard & Dhuey, 2008). Research has also indicated that inequalities related to relative age are independent of students’ socio-economic context (Burkam, LoGergo, Ready & Lee 2007; DFE, 2010). Some studies have indicated that younger students with early schooling achieve better results than those with later schooling (González-Betancor & López-Puig, 2015). In this study, school results were negatively associated with the quarter in which the student was born and positively associated with early schooling, controlling the factor “schooling time”. Other research has concluded that the age schooling begins does not itself explain the differences according to birth month, as the difference in maturity level (cognitive, social and emotional) will continue over time (Datar, 2006; Elder & Lubotsky, 2009; Huang & Invernizzi, 2013). Another variable of interest is the repetition of the school year, with evidence existing that students who repeat a year get worse results than the class average (Beebe-Frankenberger, Bocian, MacMillam, & Gresham, 2004; Ferrao, 2015). In the report published in Asturias in 2016 (Consejería de Educación y Cultura del Gobierno del Principado de Asturias, 2016) looking at repetition of school years over 15 years in OECD countries according to the estimates of PISA 2012 -Spain being one of the countries with the highest rates of repetition- it was concluded that repetition has little or no effect on academic achievement and that repeating students exhibit lower levels of motivation and confidence in their possibilities than non-repeating students and it is these which are associated with worse school results, although the study did not address birth month as a variable. Various studies (Wallingford & Prout, 2000; Gledhill, Ford & Goodman, 2002; Martin, Foels, Clanton & Moon, 2004; Polizzi, Martin & Dombrowski, 2007) have shown that the younger students in each school year are disproportionately diagnosed with developmental delays, difficulties in

162 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain language and/or mathematics, and Attention Deficit and Hyperactivity Disorder (ADHD); with each additional month of relative age, the likelihood of being diagnosed as a student with educational difficulties diminishes by 2-5% (Menet, Eakin, Stuart & Rafferty, 2000; Verachtert, De Fraine, Onghena & Ghesquière, 2010). The youngest student in the year is logically the most immature, has lower levels of confidence and self- esteem, fewer social skills, is more likely to suffer abuse at school and from behavioral problems in the classroom, regardless of the family socio- economic context (Mühlenweg, 2010; Crawford, Dearden & Meghir, 2007). Considering the fact that the expenditure on special educational needs has grown much more quickly than expenditure on general education, then if, in effect, the youngest students in a class suffer from nothing more than being born later than their classmates, categorizing a disproportionate fraction of relatively young children as candidates for repeating the year or as having learning difficulties would have significant implications for educational policies. In Spain, the ordinary measures taken for students with failing marks is the application of educational support or the possibility of an evaluation by the educational guidance team if needed in order to determine their needs.

Method

Objective

The objective of this research is to evaluate the relationship between birth month and students’ marks in language and mathematics in 2nd, 4th and 6th grade of primary education in public schools in the Principado de Asturias, Spain. It will also address the relationship between repetition of school years and birth month, and the relationship between educational support requirements and birth month.

Design

A retrospective cohort of all children born in 2002 who were in the 2nd grade of primary education in public schools in the Principado de

Revista de Educación, 386. October-December 2019, pp. 159-180 163 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

Asturias in school year 2008-2009 were followed until 2013 (6th grade of primary education) in order to analyze their results in mathematics and Spanish language and their needs for educational support.

Study population

The study population comprised the 7,063 students born in 2002 who were in 2nd grade primary in school year 2008-2009 in public schools in Asturias. Students born in other years were excluded, even if they were in 2nd grade in 2008. Students were excluded from the cohort (which completed 6th grade in 2013-2014) in 4th and 6th grades who repeated school years (students with poor marks in language and mathematics) and those who did not continue in public schools in Asturias.

Outcome measures

Student outcomes were the final marks in language and mathematics (poor, satisfactory, good, very good, and outstanding). The final marks were determined by the educational legislation in effect at the time of the study, which structures the education system in 2-year cycles and only has evaluations at the end of 2nd grade (first cycle), th4 grade (second cycle) and 6th grade (third cycle), there are no final grades in st1 , 3rd or 5th grades. The variable repeating a year was constructed with a score of poor in both language and mathematics, with a score of poor in 2nd grade and the student not appearing in 4th grade (and the same in relation to 4th and 6th grades), as they would be legally obliged to repeat the year. Students needing support to be able to pass gave us the variable Educational Support.

Other variables

Birth month (January to December), sex, school year (2nd, 4th, or 6th grade), and school.

164 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

Source of Information

Department of Education, Culture and Sport. Principado de Asturias (Spain).

Legal and ethical requirements

The Department of Education, Culture and Sport of the Principado de Asturias provided irreversibly anonymized data, which would not allow identification even when cross-checked with other databases. The data, the minimum required for the study, were used solely and exclusively for the research objectives, by people directly involved in the research, and were stored in conformance with the requirements of EU regulation 2016/679 on the protection of personal data.

Data analysis

Firstly, the distribution of students by sex and school year was described according to birth month. Following that, separately for each school year, sex, and for language and mathematics, a bivariate analysis was performed to assess the association between final marks and birth month. The χ2 test for trends was used to assess the possible statistical significance, which detects whether there is a gradient between the months of January and December. Similarly, bivariate analyses were performed to determine the number of repeats, according to birth month, and the relationship between educational support and birth month. In order to examine the independent effect of birth month on marks in each subject, considering the possible effect of school on mark distribution, the subpopulation born in the first and last quarters of the year (January to March vs. October to December) was extracted, the marks were simplified outstanding( vs. the other marks) and a multilevel logistical regression analysis was performed to examine whether the likelihood of achieving a mark of outstanding in mathematics or language differed in students born in the final quarter of the year, controlling for the effect of sex and school year, as well as the tendency for schools to

Revista de Educación, 386. October-December 2019, pp. 159-180 165 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

use outstanding marks to differing degrees. Generalized mixed linear models with logit link function were used separately for each subject. The birth month (reference: October-December), sex (reference: male), and the school year (reference: 2nd grade) were modelled as fixed effects, while the school was modelled as a random effect. All analysis was performed using R (R Core Team, 2017; R Foundation for Statistical Computing, Vienna, Austria).

Results

The cohort from 2008-2013 who attended 2nd grade primary in school year 2008-09 in public schools in Asturias was made up of a total of 7,063 pupils (Table 1), of which 49.6% were girls. The percentage of students born in the different months of the year varied between 7.5% (February) and 8.8% (December), without statistically significant differences. Nor were significant differences found in the gender makeup of students born in different months (which ranged from 46.8% girls in October to 52.5% girls in July). The cohort had 6,569 students in 4th grade (93% of the number who started 2nd grade) and 6,254 in 6th grade (88.5% of those starting 2nd grade), a total loss of 809 (11.4%) between 2nd and 6th grade due to repeating a year (failing language and mathematics) or other causes. The loss varied by birth month (from 11.0% for those born in January, 9.3% for February and 9.4% for March, to 14.5% for those born in December).

166 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

TABLE I. Distribution by birth month and student gender in public primary schools in Asturias 2008-2013 (in 2nd .4th and 6th grade).

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total Cohort in 2nd grade n 579 527 555 601 597 588 587 577 621 613 593 625 7063 % total 8.2 7.5 7.8 8.5 8.4 8.3 8.3 8.2 8.8 8.7 8.4 8.8 100.0 % girls 50.4 51.0 49.4 51.7 45.9 51.4 52.5 49.9 49.3 46.8 50.9 46.9 49.6 Cohort in 4th grade n 554 501 525 551 549 550 557 544 573 558 540 567 6569 Cohort in 6th grade n 515 478 503 527 516 523 532 517 551 544 514 534 6254 Losses from the cohort between 2nd and 6th grade n 64 49 52 74 81 65 55 60 70 69 79 91 809 % loss 11.0 9.3 9.4 12.3 13.6 11.0 9.4 10.4 11.3 11.2 13.3 14.5% 11.4

Source: authors’ own work

Figure I shows the distribution of final marks poor,( satisfactory, good, very good and outstanding) for both subjects, boys and girls, and in the different school years according to birth month. Marks of poor tend to increase by birth month (more frequent in those born in the final months of the year), whereas the outstanding marks show the opposite trend (more frequent in those born in the first few months of the year). This trend is much more marked in 2nd grade and in mathematics.

Revista de Educación, 386. October-December 2019, pp. 159-180 167 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

FIGURE I. Percentage of students from the 2008-2013 cohort in 2nd,4th, and 6th grade primary education in the Asturias with final marks in language and mathematics of poor, satisfactory, good, very good, and outstanding

O: Outstanding; VG: Very good; G: Good; S: Satisfactory; P: Poor. Source: Authors’ own work

Table II quantifies these percentages for the marks of poor and outstanding in mathematics. For 2nd grade boys, the percentage of poor marks went from 4.4% for those born in January to 10% for those born in December, while for girls the figures were 3.9% for those born in January to 10.4% for those born in December (p < 0.001 in both cases). These differences diminish markedly in 4th and 6th grades, where there is no significant trend by birth month. For outstanding marks in mathematics, the differences in 2nd grade boys ranged from 20.4% (born in December) to 36.9% (born in January), with similar differences in girls (22.1% compared to 36.7%). In this case the differences continued to be significant in th4 and 6th grade, both for boys and girls.

168 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

TABLE II. Percentage of the 2008-2013 cohort with a mark of poor or outstanding in mathematics by birth month and gender in 2nd, 4th, and 6th grades.

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total p Percentage of boys with a mark of poor in mathematics 2nd grade 4.4 5.7 5.1 7.6 8.3 6.9 5.2 5.6 8.2 7.8 9.4 10.0 7.1 0.003 4th grade 6.7 6.0 5.7 5.1 6.6 6.9 6.3 6.0 4.9 5.4 6.6 7.2 6.1 0.909 6th grade 6.8 3.5 6.3 8.0 3.7 7.2 8.5 9.0 4.7 8.9 5.6 5.4 6.5 0.585 Percentage of girls with a mark of poor in mathematics 2nd grade 3.9 3.1 4.1 6.1 6.9 5.8 4.2 7.4 6.9 8.0 8.2 10.4 6.3 <0.001 4th grade 6.6 4.9 4.7 5.0 6.6 7.5 5.7 4.2 4.5 7.4 5.4 7.8 5.9 0.547 6th grade 5.4 4.5 6.5 5.2 7.3 4.2 5.5 5.9 5.3 6.4 5.1 9.0 5.8 0.285 Percentage of boys with a mark of outstanding in mathematics 2nd grade 36.9 41.1 42.1 36.1 34.2 33.1 32.6 31.0 31.8 26.8 24.8 20.4 32.3 <0.001 4th grade 31.1 39.7 36.4 28.3 26.6 34.0 29.3 29.2 29.9 25.7 26.0 18.2 29.3 <0.001 6th grade 24.0 32.3 29.6 26.0 25.3 26.9 27.2 25.8 25.4 26.4 19.7 17.0 25.3 0.001 Percentage of girls with a mark of outstanding in mathematics 2nd grade 36.7 45.8 33.6 35.6 34.5 29.2 28.2 31.6 26.9 23.0 17.4 22.1 30.3 <0.001 4th grade 34.3 36.8 33.3 31.1 33.2 26.8 26.3 31.7 25.7 22.6 21.8 22.7 28.8 <0.001 6th grade 32.3 33.5 31.3 32.1 30.5 27.0 24.0 31.6 26.3 24.9 26.0 24.9 28.7 <0.001

Note: p corresponds to the χ2 test for trends. Source: authors’ own work

In the subject of Spanish language (Table III), the number of poor marks is lower in girls (5.8% in 2nd grade) than boys (8.4%), and girls have more outstanding marks (36.0% vs. 24.4% in 2nd grade). Significant differences are evident by month which, for poor marks, diminish in 4th and 6th grade, while outstanding marks continue to exhibit differences. A tenth (10.7%) of boys and 8.6% of girls born in December failed language in 2nd grade compared to 6.6% and 2.8% respectively of those born in January. Similarly, 16% of boys and 26.1% of girls born in December achieved outstanding marks for language, compared to 31.4% and 43.1% respectively of those born in January.

Revista de Educación, 386. October-December 2019, pp. 159-180 169 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

TABLE III. Percentage of the 2008-13 cohort with poor and outstanding marks in language by birth month and gender in 2nd, 4th, and 6th grades.

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total Percentage of boys with a mark of poor in language 2nd grade 6.6 6.1 5.1 7.9 9.6 8.7 6.7 7.4 8.2 10.1 12.2 10.7 8.4 0.001 4th grade 7.1 5.6 6.9 5.1 6.3 9.3 4.3 7.1 4.2 5.1 8.5 6.9 6.4 0.965 6th grade 8.0 5.8 4.7 6.0 5.1 8.4 4.9 8.2 4.7 6.4 7.2 5.8 6.3 0.968 Percentage of girls with a mark of poor in language 2nd grade 2.8 2.7 3.4 6.4 5.4 5.8 5.6 7.4 5.9 8.0 7.8 8.6 5.8 <0.001 4th grade 4.8 2.8 2.0 4.3 4.5 5.7 3.6 3.1 3.0 4.3 4.7 5.9 4.0 0.347 6th grade 2.7 2.9 4.9 3.0 5.6 3.5 1.5 2.0 3.4 4.8 2.8 4.9 3.4 0.710 Percentage of boys with a mark of outstanding in language 2nd grade 31.4 34.1 32.6 25.6 23.3 25.8 22.2 22.9 21.0 20.9 20.9 16.0 24.4 <0.001 4th grade 28.5 36.3 33.0 25.2 26.2 27.0 29.3 27.7 23.2 22.8 21.7 14.4 26.1 <0.001 6th grade 25.6 29.6 29.6 22.0 19.8 22.9 23.6 22.7 23.6 18.9 18.9 17.3 22.7 <0.001 Percentage of girls with a mark of outstanding in language 2nd grade 43.1 49.2 43.3 36.9 38.3 34.5 32.1 41.3 29.0 32.8 27.4 26.1 36.0 <0.001 4th grade 45.8 46.2 42.7 36.4 41.4 34.3 34.5 42.1 32.3 31.1 30.7 28.2 37.1 <0.001 6th grade 41.6 38.0 39.8 35.1 34.3 34.4 29.5 32.8 29.7 26.9 29.1 31.0 33.5 <0.001

Note: p corresponds to the χ2 test for trends. Source: authors’ own work.

Table IV shows the extra support assigned to students in the different school years, which also exhibit a gradient in 2nd grade according to birth month: from 2.1% of those born in January to 4.6% of those born in December (p<0.001). These differences do not appear in 4th or 6th grade.

170 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

TABLE IV. Percentage of the 2008-13 cohort with educational support by birth month in 2nd, 4th, and 6th grade.

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total p 2nd grade n total 579 527 555 601 597 588 587 577 621 613 593 625 7063 <0.001 n supported 12 9 12 20 22 15 15 18 29 25 37 29 243 % suppor- 2.1% 1.7% 2.2% 3.3% 3.7% 2.6% 2.6% 3.1% 4.7% 4.1% 6.2% 4.6% 3.4% ted 4th grade n total 554 501 525 551 549 550 557 544 573 558 540 567 6569 0.50 n supported 22 11 11 13 12 22 14 11 18 18 19 16 187 % suppor- 4.0% 2.2% 2.1% 2.4% 2.2% 4.0% 2.5% 2.0% 3.1% 3.2% 3.5% 2.8% 2.8% ted 6th grade n total 515 478 503 527 516 523 532 517 551 544 514 534 6254 0.44 n supported 14 10 11 11 12 11 6 10 18 16 17 15 151 % suppor- 2.7% 2.1% 2.2% 2.1% 2.3% 2.1% 1.1% 1.9% 3.3% 2.9% 3.3% 2.8% 2.4% ted

Note: p corresponds to the χ2 test for trends. Source: authors’ own work.

In the multilevel model, taking a 2nd grade boy born in the final quarter of the year as the reference (Table V, Figure II), being a girl does not significantly increase the likelihood of achievingoutstanding in mathematics, but being born in the first quarter increases it 97%. Looking at the model fit by school years shows that in th4 grade the likelihood of achieving outstanding compared to second grade is 9% lower (not statistically significant). Inth 6 grade the probability of achieving outstanding is 19% lower compared to 2nd grade, and is statistically significant. In Spanish language, in 2nd grade, being born in the first quarter of the year more than doubles the likelihood of achieving outstanding compared to those born in the last quarter of the year. Being female increases the likelihood by 83%. In 4th grade the likelihood of achieving outstanding does not change, but in 6th grade it is 13% lower. The multivariable analysis also shows that the children’s school explains 9.9% of the variability in obtaining outstanding marks in Mathematics, and 11.6% of the variability in language.

Revista de Educación, 386. October-December 2019, pp. 159-180 171 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

TABLE V. Probability of achieving outstanding in mathematics or language for those born in the first quarter vs. those born in the last quarter. Multilevel logistic regression with random effects by school.

Mathematics Language

Fixed effects OR 95%CI p OR 95%CI p

Born in first quarter 1.97 1.79 – 2.16 <0.001 2.05 1.87 – 2.25 <0.001

4th grade 0.91 0.82 – 1.02 0.095 1.01 0.91 – 1.13 0.823

6th grade 0.81 0.73 – 0.91 <0.001 0.87 0.78 – 0.98 0.018

Sex (female) 1.05 0.95 – 1.15 0.335 1.83 1.67 – 2.01 <0.001

Constant 0.25 0.22 – 0.29 <0.001 0.19 0.16 – 0.22 <0.001

Random effects

τ00. school 0.363 0.430

Nschool 292 292

ICCschool 0.099 0.116

Note: n: 10097; Deviance mathematics: 10900.089; Deviance language: 10914.900. OR: Odds ratio; 95%CI: 95% Confidence Interval for OR; ICC: Intraclass correlation coefficient. Source: Authors’ own work

172 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

FIGURE II. Probability of achieving outstanding in language (top) and mathematics (bottom) for those born in the first quarter compared to those born in the last quarter.

Source: authors’ own work

Discussion and conclusions

The results of our study are consistent with research which has examined the impact of birth month on school achievement, confirming that in our context students with lower relative ages are more likely to achieve worse results. The broader causal argument, brought together in the cited research to explain this effect, is that students who are more immature than the class average face greater challenges at school than their older classmates (Hutchison & Sharp, 1999; Plewis, Canderwood, Hawkes, Hughes & Joshi, 2007; Sykes, Bell & Rodeiro, 2009; Wattie, 2013; Department for Education, 2010). Given that most year repeats and diagnoses of special needs are made in the first years of schooling, where the impact of relative age is greatest,

Revista de Educación, 386. October-December 2019, pp. 159-180 173 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

and with birth month not being a variable that is considered when it comes to adapting tests or adjusting results, it is difficult to distinguish the extent to which school failure is due more to a lack of maturity than to a lack of ability (Goodman, Gledhill & Ford, 2003; Elder & Lubotsky, 2009). The results of this study show a clear association between relative age and worse school results, both in mathematics and Spanish language, and in both boys and girls. These results are particularly notable in 2nd grade but they continue (or are transferred) in later school years. The percentages of repeats of year is also higher for those born in the last few months of the year. The same occurs for the need for educational support, although in this case, it does not seem to transfer to later school years. Different studies have proposed various measures to compensate for the impact of relative age. Among them are flexible student groupings (Kutnick, Sebba, Blatchford, Galton & Thorp, 2005; Blatchford, Hallam, Ireson, Kutnick & Creech, 2008; Campbell, 2013) such as tests being given according to birth month or failing that, modifications in test content. Some authors have recommended applying alternative systems to evaluate learning (which respect learners’ rates of learning and learning styles) and adjusting results according to birth month so that they are fairer for all students, and so that younger students are not predestined to have worse results simply because of the day they were born (Smith, 2010; Crawford, Dearden & Greaves, 2013). Given that in the Spanish context, apart from a few exceptions (Gonzalez-Vallinas, Librero, Peiró & San Fabián, 2018), the research into the impact of relative age on school results has been very limited, new studies with other cohorts in other autonomous communities will be needed to confirm the potential generalizability of our results from Asturias. We did not have qualitative data to allow us to to analyze the negative effect of relative age on the self-esteem or emotional development of the youngest children in the class, students who spend all of their primary schooling not only getting the worst marks but also receiving daily negative feedback (as the slowest, the clumsiest, etc.), and a higher likelihood of being classified as students with difficulties, and having special educational needs. Research is needed which looks at the impact of this on children’s psycho-affective development and possible

174 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain diagnoses related to birth month, as other studies have done (Wallingford & Prout, 2000; Martin, Foels, Clanton & Moon, 2004; Polizzi, Martin & Dombrowski, 2007; Dhuey & Lipscomb, 2010).

References

Allen, J., & Barnsley R. (1993). Streams and Tiers: The Interaction of Ability, Maturity, and Training in Systems with Age-Dependent Recursive Selection. The Journal of Human Resources, 28(3), 649-659. DOI:10.2307/146164. Bedard, K. & Dhuey, E. (2006). The Persistence of Early Childhood Maturity: International Evidence of Long-Run Age Effects. The Quarterly Journal of Economics, 121(4), 1437-1472. DOI: 10.1093/ qje/121.4.1437. Bedard, K. & Dhuey, E. (2008). Is September better than January? The Effect of School Entry Age Laws on Skill Accumulation. Working Paper. Department of Economics, University of California, Santa Barbara. Retrieved from https://www.researchgate.net/publication/289963418_ Is_September_Better_than_January_The_Effect_of_School_Entry_ Age_Laws_on_Skill_Accumulation. Beebe-Frankenberger, M., Bocian, K.M., MacMillam, D.L. & Gresham, F.M. (2004). Sorting second-grade students: Differentiating those retained from those promoted. Journal of Educational Psychology, 96(2), 204- 215. DOI: 10.1037/0022-0663.96.2.204. Black, S., Devereux, P.J. & Salvanes, K.G., (2008). Too Young to Leave the Nest? The Effects of School Starting Age. The Review of Economics and Statistics, 93(2), 455–467. Blatchford, P., Hallam, S., Ireson, J. Kutnick, P. & Creech, A. (2008) Classes, Groups and Transitions: structures for teaching and learning. University of Cambridge. Retrieved from Retrieved from https://cprtrust.org.uk/ wp-content/uploads/2014/06/research-survey-9-2.pdf. Boardman, M. (2006) The impact of age and gender on Prep children’s academic achievements. Australasian Journal of Early Childhood, 31(4), 1-6. DOI: 10.1177/183693910603100402.

Revista de Educación, 386. October-December 2019, pp. 159-180 175 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

Burkam, D.L., LoGergo, L., Ready, D. & Lee, V. (2007). The Differential Effects of Repeating Kindergarten. Journal of Education for Students Placed at Risk, 12(2), 103-136. Campbell, T. (2013). In-school ability-grouping and the month of birth effect: Preliminary evidence from the Millennium Cohort Study. CLS Cohort Studies Working Paper No. 2013/1. London: Institute of Education, University of London. Retrieved from https://cls.ucl.ac.uk/ wp-content/uploads/2017/04/Ability-grouping-and-the-month-of- birth-effect-T-Campbell-March-2013-FINAL.pdf. Consejería de Educación y Cultura del Gobierno del Principado de Asturias. Dirección General de Ordenación académica e innovación educativa. Servicio de Evaluación educativa. (2016). Informe de Evaluación nº 2. La repetición escolar: hechos y creencias. Consejería de Educación y Cultura. Retrieved from https://www.educastur.es/ documents/10531/879356/2016-09+informes+evaluación+repetición. pdf/6c1ddfcc-3146-4d99-b85c-85ad90a78345. Consejería de Educación, Cultura y deporte. Gobierno de Cantabria. (2014) Evaluación de diagnóstico 2012-2013. Retrieved from http:// www.educantabria.es/docs/Avance_informe_curso_2012-13_v1.pdf. Crawford, C., Dearden, L., & Meghir, C. (2007). When You Are Born Matters: The Impact of Date of Birth on Child Cognitive Outcomes in England. Centre for the Economics of Education. London School of Economics. Retrieved from https://eprints.lse.ac.uk/19374/1/When_ you_are_born_matters.pdf. Crawford, C., Dearden, L. & Greaves, E. (2011). Does when you are born matter? The impact of month of birth on children’s cognitive and non-cognitive skills in England. IFS Working Paper W10/06. London: Institute for Fiscal Studies. Retrieved from http://www.ifs.org.uk/bns/ bn122.pdf. Crawford, C., Dearden, L. & Greaves, E. (2013). When you are born matters: evidence for England. IFS Report 80. London: Institute for Fiscal Studies. Retrieved from http://www.ifs.org.uk/comms/r80.pdf. Daniels, S., Shorrocks-Taylor, D. & Redfern, E. (2000). Can starting summer-born children earlier at infant school improve their national curriculum results? Oxford Review of Education, 26(2), 207-220. Datar, A. (2006). Does Delaying Kindergarten Entrance Give Children a Head Start? Economics of Education Review, 25(1), 43–62. DOI: 10.1016/j.econedurev.2004.10.004.

176 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

Department for Education. (2010). Month of Birth and Education: School Analysis and Research División. Department for Education, UK Government. Retrieved from https://dera.ioe.ac.uk/835/1/DFE- RR017.pdf. Dhuey, E. & Lipscomb, S. (2010). Disabled or Young? Relative Age and Special Education Diagnoses in Schools. Economics of Education Review, 29, 857–872. DOI: 10.1016/j.econedurev.2010.03.006. Elder, T.E. & Lubotsky, D.H. (2009). Kindergarten Entrance Age and Children’s Achievement: Impacts of State Policies, Family Background, and Peers. Journal of Human Resources, 44(3), 641-683. DOI: 10.3368/ jhr.44.3.641. Ferrao, M.E. (2015). Tópicos sobre retenção escolar em Portugal através do PISA: qualidade e equidade. Arquivos Analíticos de Políticas Educativas, 23(114), 1-22. DOI: 10.14507/epaa.v23.2091. Gledhill, J., Ford, T. & Goodman, R. (2002). Does season of birth matter? The relationship between age within the school year (season of birth) and educational difficulties among a representative general population sample of children and adolescents (aged 5-15) in Great Britain. Research in Education, 68, 41-47. DOI: 10.7227/RIE.68.4. González-Bentacor, S.M., & López-Puig, A.J. (2015). Escolarización temprana, trimestre de nacimiento y rendimiento educativo en Primaria. Revista de educación, 369, 159-183. DOI: 10.4438/1988- 592X-RE-2015-369-294. González Vallinas, P., Librero, J., Peiró, S. & San Fabián, J.L. (2018). Edad Relativa y Resultados Escolares en Educación Primaria en la Comunidad Autónoma de Cantabria. Archivos Analíticos de Políticas Educativas, 26(141), 1-20. DOI: 10.14507/epaa.26.2926. Goodman, R., Gledhill, J. & Ford, T. (2003). Child Psychiatric Disorder and Relative Age Within School Year: Cross Sectional Survey of Large Population Sample. British Medical Journal, 327(7413), 472. DOI: 10.1136/bmj.327.7413.472. Huang, F.L. & Invernizzi, M.A. (2013). Birthday Effects and preschool attendance. Early Childhood Research Quarterly, 28(1), 11-23. DOI: 10.1016/j.ecresq.2012.03.002. Hutchison, D. & Sharp, C. (1999). A lasting legacy? The persistence of season of birth effects. National Foundation for Educational Research. Retrieved from https://www.nfer.ac.uk/media/1315/44411.pdf.

Revista de Educación, 386. October-December 2019, pp. 159-180 177 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

Kawaguchi, D. (2006). The effect of age at school entry on education and income. ESRI Discussion Paper Series No 162. Tokio: Economic and Social Research Institute. Retrieved from http://www.esri.go.jp/ jp/archive/e_dis/e_dis170/e_dis162.pdf. Kinard, E.M. & Reinherz, H. (1986). Birthdate effects on school performance and adjustment: A longitudinal study. The Journal of Educational Research, 79(6), 366-372. DOI: 10.1080/00220671.1986.10885707. Kutnick, P., Sebba, J., Blatchford, P., Galton, M. & Thorp, J. (2005). The effects of pupil grouping: Literature review. Department for Education and Skills, University of Brighton. Retrieved from https://core.ac.uk/ download/pdf/15171799.pdf. La Paro, K.M. & Pianta R.C. (2000). Predicting children’s competence in the early school years: A meta-analytic review. Review of Educational Research, 70 (4), 443-484. DOI: 10.3102/00346543070004443. Lawlor, H., Clark, H., Ronalds, G. & Leon, D. (2006). Season of birth and childhood intelligence: findings from the Aberdeen Children of the 1950s cohort study. British Journal of Educational Psychology, 76(3), 481-499. DOI:10.1348/000709905X49700. Martin, R. P., Foels, P., Clanton, G. & Moon, K. (2004). Season of birth is related to child retention rates, achievement, and rates of diagnosis with specific LD. Journal of Learning Disabilities, 37(4), 307-317. DOI: 10.1177/00222194040370040301. Martín, E., Martínez, R., Marchesi, A. & Pérez, E.M. (2008). Variables that predict academic achievement in the Spanish compulsory secondary educational system: a longitudinal, multi-level analysis. The Spanish Journal of Psychology, 11 (2), 400-413. Menet, F., Eakin, J., Stuart, M. & Rafferty, H. (2000). Month of Birth and Effect on Literacy, Behaviour and Referral to Psychological Service. Educational Psychology in Practice, 16(2), 225-234. DOI: 10.1080/713666055. Mühlenweg, A. (2010). Young and innocent: international evidence on age effects within grades on victimization in elementary school. Economics Letters, 109(3), 157-60. DOI: 10.1016/j.econlet.2010.08.032. Navarro, J.J., García-Rubio, J. & Olivares P.R. (2015). The Relative Age Effect and Its Influence on Academic Performance. Plos One, 10(10), e0141895. DOI:10.1371/journal.pone.0141895. Oshima, T.C. & Domaleski, C.S. (2006). Academic Performance Gap Between Summer-Birthday and Fall-Birthday Children in Grades K-8.

178 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

The Journal of Educational Research, 99(4), 212-217. DOI: 10.3200/ JOER.99.4.212-217. Plewis, I., Canderwood, L., Hawkes, D., Hughes, G. & Joshi, H. (2007). Millennium Cohort Study: Technical Report on Sampling (4th Edition). Centre for Longitudinal Studies, Bedford Group for Lifecourse and Statistical Studies, Institute of Education, University of London. Retrieved from https://sp.ukdataservice.ac.uk/doc/5350/mrdoc/pdf/ mcs_technical_report_on_sampling_4th_edition.pdf. Polizzi, N., Martin, R.P. & Dombrowski, S.C. (2007). Season of birth of students receiving special education services under a diagnosis of emotional and behavioural disorder. School Psychology Quarterly, 22(1), 44-57. DOI: 10.1037/1045-3830.22.1.44. Puhani, P.A. & Weber, A.M. (2007). Does the Early Bird Catch the Worm? Instrumental Variable Estimates of early Educational Effects of Age of School Entry in Germany. Empirical Economics, 32(2-3), 359-386. DOI: 10.1007/s00181-006-0089-y. Robertson, E. (2011). The effects of quarter of birth on academic outcomes at the elementary school level. Economics of Education Review, 30(2), 300-311. DOI: 10.1016/j.econedurev.2010.10.005. Matta, R., Ribas, R.P., Sampaio, B. & Sampaio, G.R. (2016). The effect of age at school entry on college admission and earnings: a regression- discontinuity approach. IZA Journal of Labor Economics, 5(9), 1-25. DOI: 10.1186/s40172-016-0049-5. Sharp, C., George, N., Sargent, C., O’Donnell, S. & Heron, M. (2009). International thematic probe: The influence of relative age on learner attainment and development. National Foundation for Educational Research. Retrieved from https://assets.publishing.service.gov. uk/government/uploads/system/uploads/attachment_data/ file/604978/0209_CarolineSharp_et_al_RelativeAgeReviewRevised.pdf. Smith, J. (2010). How Valuable Is the Gift of Time? The Factors That Drive the Birth Date Effect in Education. Education Finance and Policy, 5(3), 247-277 DOI: 10.1162/EDFP_a_00001. Sprietsma, M. (2010). The effect of relative age in the first grade of primary school on long-term scholastic results: International comparative evidence using PISA 2003. Education Economics, 18(1), 1-32. DOI: 10.1080/09645290802201961. Strom, B. (2004). Student achievement and birthday effects. Department of Economics. Trondheim: Norwegian University of Science and

Revista de Educación, 386. October-December 2019, pp. 159-180 179 Received: 20-07-2018 Accepted: 23-07-2019 González-Vallinas, P., Librero, J., Peiró, S., San Fabián, J.L. The impact of relative age on results in language and mathematics in primary education in Asturias, Spain

Technology. Retrieved from http://www.hks.harvard.edu/pepg/PDF/ events/Munich/PEPG-04-24Strom.pdf. Sykes, E.D.A., Bell, J.F. & Vidal Rodeiro, C.V. (2009). Birthdate Effects: A Review of the Literature from 1990-on. Cambridge Assessment, University of Cambridge. Retrieved from https://www. cambridgeassessment.org.uk/Images/109784-birthdate-effects-a- review-of-the-literature-from-1990-on.pdf. Thoren, K., Heinig, E. & Brunner, M. (2016). Relative Age Effects in Mathematics and Reading: Investigating the Generalizability across Students, Time and Classes. Frontiers in Psychology, 7, 679. DOI: 10.3389/fpsyg.2016.00679. Verachtert, P., De Fraine, B., Onghena, P. & Ghesquière, P. (2010). Season of birth and school success in the early years of primary education. Oxford Review of Education, 36(3), 21. DOI: 10.1080/030549 81003629896. Wallingford, E.L. & Prout, H.T. (2000). The relationship between season of birth and special education referral. Psychology in the Schools, 37(4), 379-387. DOI: 10.1002/1520-6807(200007)37:4<389:AID- PITS9>3.0.CO;2-K. Wattie, N. (2013). Relative age effects in education and sport: An argument for human, not statistical solutions. Radical Statistics, (108), 5-20. Retrieved from http://www.radstats.org.uk/no108/Wattie108.pdf.

Contact Address: Paula González-Vallinas. E.O.E. Avilés. Consejería de Educación y Cultura. Asturias. E.O.E. Avilés. Consejería de Educación y Cultura. Asturias. E-mail: [email protected].

180 Revista de Educación, 386. October-December 2019, pp. 159-180 Received: 20-07-2018 Accepted: 23-07-2019 Exploratory study on bullying in gifted students: prevalence and psychological wellbeing1

Estudio exploratorio sobre acoso escolar en alumnado con altas capacidades: prevalencia y afectación psicológica

DOI: 10.4438/1988-592X-RE-2019-386-432

Joaquín González-Cabrera Javier Tourón Juan Manuel Machimbarrena Ana León-Mejía Mónica Gutiérrez-Ortega Universidad Internacional de La Rioja

Abstract Introduction: The distinguishing characteristics of gifted students can leave them vulnerable to being bullied. Evidence on school violence and gifted students is scarce and it has still not been explored in Spain. The aim is to establish the prevalence of bullying among gifted students, its distribution across the different roles and its relationship with other psychological variables. Method: We employed a cross-sectional and analytic study of 285 gifted students

(Mage = 11.77; SD = 2.28; range 9–18) from all over Spain (175 male, 61.6%). The Spanish versions of the European Bullying Intervention Project Questionnaire, Depression, Anxiety, Stress Scales-21, Interpersonal Support Evaluation List and KIDSCREEN-10 were used. Results: 50.9% is involved in bullying, with 36.9% being victims, 1.1% bullies, and 10.2% bully-victims. Bully-victims and victims present worse health-related quality of life scores (p < .001), depression (p < .001), stress (p < .001) and, and anxiety (p < .001) than uninvolved individuals.

(1) This research has been partially funded by the Universidad Internacional de La Rioja under its Own Research Plan 4 [2017-2019]. It has also been funded by the project RTI2018-094212-B-I00 by the Ministry of Science, Innovation and Universities of Spain.

Revista de Educación, 386. October-December 2019, pp. 181-206 181 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

Discussion: This paper provides empirical evidence of the high prevalence of victimization among gifted Spanish students. Being a victim or bully-victims has an impact on psychological wellbeing and, particularly, on health-related quality of life, stress, anxiety, and depression.

Key words: bullying, victim, bully, gifted students, health-related quality of life, stress, depression.

Resumen Introducción: Las características diferenciales del alumnado de altas capacidades pueden hacerlo vulnerable al acoso. La evidencia empírica sobre violencia escolar y altas capacidades es escasa y aún no explorada en el contexto español. El objetivo principal es conocer la prevalencia del bullying, su distribución en los diferentes roles y la relación con otras variables psicológicas.

Metodología: Estudio transversal y analítico con 285 alumnos (Mage=11.77; SD=2.28; range 9-18) con altas capacidades de todo el territorio español (175 varones, 61.6%). Se utilizaron las versiones españolas del European Bullying Intervention Project Questionnaire, Depression, Anxiety, Stress Scales-21, Interpersonal Support Evaluation List y KIDSCREEN-10. Resultados: El 50.9% está implicado en el acoso escolar, siendo el 39.6% víctimas, el 1.1% agresores y el 10.2% víctimas-agresoras. Las víctimas-agresoras y las víctimas presentan peores puntuaciones en calidad de vida relacionada con la salud (p<.001), depresión (p<.001), estrés (p<.001) y ansiedad (p<.001) que los no implicados. Discusión: Este trabajo proporciona evidencia empírica sobre la alta prevalencia de la victimización entre los estudiantes de altas capacidades españoles. Ser una víctima o una víctima-agresora muestra un impacto en el bienestar psicológico y particularmente cuando se trata de calidad de vida relacionada con la salud, estrés, ansiedad y depresión.

Palabras clave: acoso, víctima, agresor, altas capacidades, calidad de vida relacionada con la salud, estrés, depresión.

Introduction

Bullying: Prevalence and Relationship with Psychosocial Problems

Peer intimidation and, specifically, school bullying is a significant psychosocial problem with a high impact on students and their families. The most widely accepted and commonly used definition of bullying is the

182 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing one formulated by Olweus (1999), according to which a student is being intimidated when another student (or group of students) deliberately act (physically, verbally, socially or psychologically) with the intention of causing her or him suffering over an extended period of time. Bullying has a high prevalence that appears to have increased over the recent years (Aboujaoude, Savage, Starcevic & Salame, 2015). This is shown in the work of Modecki, Minchin, Harbaugh, Guerra and Runions (2014), which summarizes over 80 studies, putting the rate of prevalence of bullying at 35% for perpetration and 36% for victimization. In the case of Spain, several reviews such as those by Garaigordobil and Martínez- Valderrey (2014) and Zych, Ortega-Ruiz and Marín-López (2016) report both traditional bullying and cyberbullying in all regions studied. The most recent Spanish study, with a sample of over 20,000 students, places victimization at 9.3% (Save the Children, 2016). Likewise, the largest study of bullying performed in Spain, with almost 26,000 participants, identifies 11.7% of victimization and 4% of perpetration (González- Cabrera, Balea, Vallina, Moya & Laviana, 2017). As is to be expected, results vary depending on the age of participants, assessment tools, research designs, cut-off points and analyses implemented (Berne et al., 2013; Zych et al., 2016). It has been pointed out that school violence seems to diminish as age increases, and that the greatest peak in prevalence seems to occur between the ages of 11 and 14, with boys displaying more aggressive behavior than girls (Garaigordobil, Martínez-Valderrey & Aliri, 2014; Pérez-Fuentes, Gázquez, Molero, Soler & Barragán, 2015); however, these figures are still being debated (Zych, Ortega-Ruiz & Del Rey, 2015; Zych et al., 2016). In addition, chronic victimization usually continues over time, ranging from 8% to 43% (Pouwels, Souren, Lansu & Cillessen, 2016) and bully-victim seems to be the most prevalent profile (Festl, Vogelgesang, Scharkow & Quandt, 2017). Although all participants may suffer consequences of bullying, the roles of victim and victim-aggressor are associated with the most relevant psychological consequences (Zych et al., 2015). Specifically, being a victim of bullying is associated with worsening school performance (Nakamoto & Schwartz, 2009), a decrease of peer support, a negative impact on self-concept, and increased levels of loneliness and anxiety (Hawker & Boulton, 2000). Furthermore, it raises up to three times the chance of having worse health-related quality of life (HRQoL) compared to students who have not been bullied (Hidalgo-Rasmussen et al., 2015),

Revista de Educación, 386. October-December 2019, pp. 181-206 183 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

as well as to suffering a great deal of somatization problems (Gini & Pozzoli, 2013). According to Hawker and Boulton’s meta-analysis (2000), one of the most worrying consequences of bullying victimization is that it can lead to depression, and this path has been supported by other findings showing a relationship between bullying victimization, depression and other internalizing disorders (Reijntjes, Kamphuis, Prinzie, and Telch, 2010), and even to suicidal ideation (Holt et al., 2015). Interpersonal support has also been identified as a factor associated with bullying. In this sense, Demaray and Malecki (2003) noted that victims and bully-victims had less support than classmates who were not involved in bullying. Similarly, Holt and Espelage (2007) also found that victims had less interpersonal support than those who were bullies or were not involved in a bullying situation.

Gifted Students

Gifted students make a heterogeneous group, but they have outstanding levels of aptitude or competence in at least one area or field (Gagné, 2015). The most consolidated research approaches in the field of giftedness are currently aligned with a developmental approach that disagrees with the use of rigid cut-off point or the IQ (usually 130) as a criterion. The problem of giftedness is neither a “state of being” or a “trait” (Renzulli & Reis, 2018), nor a question of “being or not being” (cfr. p.e. Olszewski-Kubilius, Subotnik,& Worrel, 2015; Gagné, 2015). In fact, some authors indicate that the term gifted is simply a social construct (Pfeiffer, 2018). However, in Spain the identification criteria, when they exist, are diverse within the regions, although the most common ones are in line with the aforementioned IQ criterion or with the model of profiles developed by Castelló and Battle (1998). In this study, we consider any student as gifted if h/she was officially identified by the mechanism established in his or her region. Often, this group of children not only shows quantitative differences in performance when compared with same age peers, but also qualitative differences, meaning that they differ to a greater or lesser extent in aspects such as an unusual degree of sensitivity, perfectionism, greater ethical and moral awareness, close bonds, early development of an internal

184 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing locus of control, motivation, passion for learning, outstanding academic performance, etc. (Clark, 2012; Peterson, 2009; Silverman, 2002). Along with the usual challenges that go with giftedness, these features make them appear more vulnerable, as a potential target, than those who do not stand out from the group (Pelchar & Bain, 2014). However, a recent systematic review has found that this group has cognitive abilities that help them better deal with social and emotional challenges, thus contributing to lower participation in antisocial behavior and bullying victimization (Francis, Hawes & Abbott, 2016).

Bullying and Gifted Students

Empirical evidence about school violence and gifted students in Spain is non-existent, and at the international level it is scarce and has provided us with mixed results (Smith, Dempsey, Jackson, Olenchak & Gaa, 2012). Peterson and Ray (2006a) found that gifted students have a high prevalence of bullying victimization (67%), with the largest peak in sixth grade, thus showing that this is a group at risk of victimization (Peterson & Ray, 2006b). Similarly, Dalosto and Alençar (2013) reported that gifted students suffered from bullying and, less frequently, also bullied others. In contrast, a number of studies suggest that rates of bullying of gifted children are similar to that of the general population (Oliveira & Barbosa, 2011; Parker, 2011; Peters & Bain, 2011). And it has even been suggested that gifted children are less involved in bullying (Erwin, 2015; Estell et al., 2009). Finally, Ryoo, Wang, Swearer and Park (2017) find no differences in victimization between gifted students and the general population but do find an increase in the prevalence of perpetration over time in gifted students. More specifically, they become bullies after the transition from primary to secondary school more frequently than do the rest of students. Regarding the theoretical-conceptual argument that may support the higher level of victimization among gifted students, it is important to note that they are a minority group, and that it is precisely this minority vs. majority opposition what underlies gifted children’s tendency to be bullied (Zych, et al., 2015). This could be explained in terms of group socialization theory (Harris, 1995) and social identity theory (Haslam, Ellemers, Reicher, Reynolds, & Schmitt, 2010), which state that intergroup processes lead individuals to behave in a way that favors their

Revista de Educación, 386. October-December 2019, pp. 181-206 185 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

own group (in-group) and discriminates against others (out-group). The effects of these group dynamics are crucial for developing norms and constructing identities, and are typically based on the most outstanding characteristics that distinguish them. This is something that Tajfel (1982) referred as the “minimal group paradigm”, e.g., it makes it possible to justify intimidation and aggression simply because a schoolmate belongs to an out-group (Hymel, Mclure, Miller, Shumka & Trach, 2015; Reijntjes et al., 2013). Also, being labeled as gifted in the school context affects students’ socialization and peer relationships (Margolin, 2018) and is also seen by parents as a potential source of discrimination (Matthews, Ritchotte, & Jolly, 2014), as it generates rejection of those who are not labeled as gifted. This effect of distancing oneself from the out-group has even been found in 4 and 5-year-old children (Oostenbroek & Over, 2015). The characteristics and features of gifted students increase, in the current social setting, the chances of them being seen as people who do not fit in, making them a potential target of bullying. This phenomenon has been reported in other studies of minority groups (Albdour & Krouse, 2014; Zych et al., 2015). In light of this situation, the main aim of this work is to establish the prevalence of bullying among Spanish gifted students and their participation in the different roles. Other goals are: 1) to establish whether there are gender and age differences in the prevalence of bullying among gifted students; 2) to analyze the impact on students’ perceived quality of life according to the role they play in bullying; and 3) to link bullying roles to psychological variables related to wellbeing and mental health (anxiety, depression, stress, satisfaction with life and perceived interpersonal support) among gifted students. The starting hypotheses for this work are: a) The role of victim will be the more prevalent among gifted students (Peterson & Ray (2006ab). b) The victimization rates of gifted children will be greater than those found in other studies focused on general sample using the same tool (Zych et al., 2015). c) The negative psychological impact on victims and bully-victims will be higher than on bullies or on uninvolved individuals (Zych et al., 2015).

186 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

d) Gender and age differences will be similar to those found in other general student population studies (González-Cabrera et al., 2017; Save the Children, 2016).

Method

Design and Participants

An analytic and cross-sectional study was performed between June and October 2017. The total number of participants was initially 323 adolescents and after the initial screening of the database, it comprised 285 adolescents identified as gifted (175 male, 61.6%, and 109 females, 38.4%). Fueron excluidos aquellos participantes que iniciaron la batería de cuestionarios, pero no finalizaron el proceso de respuesta. The mean and standard deviation of the age was 11,77±2.28 with an age range of 9–18 years. 122 students (43.7%) were attending primary school (grades 1 to 6), 118 (42,3%) were in compulsory secondary education (grades 7 to 10), and 39 of them (14%) were taking the Spanish Baccalaureate (grades 11 and 12).

Population and Sample

The population of gifted students in Spain, as happens in many other countries, has peculiarities that should be noted. These include geographic dispersion, disparity of identifying criteria and lack of systematic detection processes that leads to under-identification, among others (Tourón & Tourón, 2016). According to the Spanish Ministry of Education there were approximately 23.745 gifted students at the beginning of 2017. This is why it is necessary to bear in mind that in Spain it is not possible to follow a strict random sampling procedure. Therefore, it is not possible to guarantee that the sample is representative, statistically speaking, of the total population of students identified. Nevertheless, the number of participants who responded is large enough to give us valuable information regarding the prevalence of bullying in this group. If the number of participants who participated in this study

Revista de Educación, 386. October-December 2019, pp. 181-206 187 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

could have been selected randomly, the margin of error for proportions with a confidence intervalα ( = 0.05) would be 5.5% for the estimate of proportions. The researchers initially contacted 40 official parent associations of gifted students located across the country. Twenty-two of these associations agreed to participate and to inform their members of this study (particularly to the legal guardians of the minors to be interviewed). As a result, we gathered data from 35 Spanish provinces and from all regions but Ceuta and Melilla. In a second stage, the people running the associations were sent the necessary documentation to participate in the study to pass it on to their members (legal guardians of the gifted minors).

Procedure

The questionnaires were administered online through a website created specifically for this study [hidden here for blind review]. In it, the legal guardians were informed of who the promoters of the research were, the research activity of the groups, the purpose of the study, its assessment tools, time frame, etc. Before accessing the online questionnaires, the legal guardians had to confirm that they had been informed and agree to their children’s participation in the study. The Survey Monkey® online platform was used, and the average amount of time taken to complete the questionnaires was approximately 25 minutes. The legal guardians and minors who participated did so voluntarily, anonymously and without any payment. The project was approved by the Research Ethics Committee of the Principality of Asturias (Ref.41/17). The criteria for inclusion in the study were as follows: a) to have an official diagnosis as gifted student; this implies having been identified according to the criteria each region, usually having an IQ equal to or greater than 130 or, in some cases, being above the 75th percentile in a battery of differential aptitudes (commonly, the BADYG, Yuste, 1989); b) to be a member of one of the associations for gifted students officially registered in Spain, c) to be of school age, more specifically, between grade 5 of primary school and grade 12 (Spanish baccalaureate) and d) to access the online battery of questionnaires with the prior permission of his/her legal guardian. There were no exclusion criteria. As this was an online process, and there is no record of the exact number of people

188 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing who received or read the invitation emails, so a concrete rejection figure cannot be given.

Instruments

The participants started by answering different questions about sociodemographic variables such as their gender, age, school grade, the association their families are members of, and the province they live in. The psychological assessment tools used are described below: a) The Spanish version of the European Bullying Intervention Project Questionnaire (EBIP-Q; Ortega, Del Rey, & Casas, 2016). This comprises 14 items, 7 describing victimization and 7 describing aggression. For both dimensions the items refer to actions such as: hitting, insulting, threatening, stealing, swearing, excluding people or spreading rumors, etc. (e.g. I have been excluded or ignored by my peers/ I have excluded or ignored a peer). It follows a Likert scale that goes from 0 (never) to 4 (always). Therefore, the scoring ranges from 0 to 28 points for each dimension. In order to establish the different involvement roles, we followed the criteria established by the authors of the questionnaire (Brighi et al., 2012). Thus, to determine the victim role, we considered the individuals who obtained scores equal to or higher than 2 in any of the victimization items. Involvement in the role of bully was calculated by taking into account the individuals who obtained scores equal to or higher than 2 in any of the aggression items. The role of bully-victim was calculated by considering the individuals who obtained scores equal to or higher than 2 in at least one item of both the aggression and victimization scales. These criteria were based on the scores of participants, the nature of the phenomenon and previous studies with this assessment tool (Brighi et al., 2012). It has also been used in sample studies with students with special educational needs (Rodriguez-Hidalgo, Alcívar, & Herrera-López, 2019). The EBIP-Q shows appropriate indicators of internal reliability and validity. The Cronbach’s alpha coefficients for the study sample were .89 and .70 for the victimization and aggression dimensions, respectively. The respondents were asked about behavior that occurred since the

Revista de Educación, 386. October-December 2019, pp. 181-206 189 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

beginning of the school year (approximately 7 months). An extra question for bystanders was added based on the Participant Role Scale (Salmivalli, Lagerspetz, Björkqvist, Österman & Kaukiainen, 1996), which make it possible to classify bystanders into different categories: a) associates of the bully (never initiates aggression but sometimes participates by supporting the bully); b) reinforcer of the bully (sympathizes with the bully, but never directly participates with him/her); c) bystander (remains neutral when there is an aggression); d) pro-victim (supports the victim, but does nothing to avoid the aggression and e) defender (usually actively defends the victim and helps him/her as much as possible). b) Spanish version of the Depression, Anxiety, Stress Scales-21 (DASS- 21). This is a self-report survey that evaluates negative emotional states over the previous week. It comprises 21 statements that are answered using a Likert scale with response options distributed over three subscales with seven items each: depression, anxiety and stress. We used the version adapted to Spanish and validated by Daza, Novy, Stanley and Averill (2002). In this study, Cronbach’s alpha for the dimension of depression was .90, for anxiety .86, and for stress .90. c) The Spanish version of the Interpersonal Support Evaluation List (ISEL), the Escala de Apoyo Social Percibido (EASP). This questionnaire evaluates the person’s view of which interpersonal support resources are potentially available. It comprises four subscales, although only the appraisal and belonging ones were used in this study. The version we used is the one adapted to the Spanish population by Trujillo, Martos and González-Cabrera (2012), which has acceptable indicators of reliability and validity. The Cronbach’s alphas for the two dimensions were .72 and .73 respectively. d) Spanish version of KIDSCREEN-10 (KIDSCREEN Group Europe, 2006). This questionnaire evaluates Health related quality of life (HRQoL) as perceived by children and adolescents aged from 8 to 18, from the WHO’s global perspective, that is from a bio-psychosocial perspective. It has appropriate levels of reliability, internal validity, and it was also normed for our population. Cronbach’s alpha was .88 for this sample.

190 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

Data Analysis

The following analyses were performed: 1) testing the assumption of normality for the variables involved in the study (the Shapiro-Wilk test) as well as the homogeneity of variances for the comparisons between groups (Levene test); 2) frequency analysis and analysis of central tendency and dispersion measures; 3) χ2 test for comparing proportions, and Student’s t test for independent samples to compare significant differences of means. In cases where statistically significant differences were found, Cohen’s d was calculated to provide an estimate of the effect size of the difference; 4) partial correlations controlling by age; 5) analysis of variance with Games-Howel post-hoc comparisons; level of significance was established for a p value below .05. The statistical analyses were performed using the “Statistical Package for the Social Sciences” (SPSS) 23 program (IBM®).

Results

Descriptors and Differences by Gender and Age

There are significant gender differences in the total scores for aggression (male 2.15±1.88, female 1.4±1.68; t = 3.415; p < .001; d = .43) and for victimization (male 6.71±5.58, female 5.5±4.68; t = 1.191; p = .049, d = .23). However, significant gender differences were not found in DASS- 21 in the stress (p = .567), anxiety (p = .944) and depression (p = .884) dimensions, and this is also the case with the ISEL in appraisal (p = .777) and belonging (p = .296). The value for HRQoL (KIDSCREEN-10) is p = .981. When considering age, we find significant differences in depression

(F9,272 = 2.216; p = .021), appraisal (F9,272 = 2.785; p = .004) and KIDSCREEN-10

(F9,272 = 3.719; p < .001). The Games-Howel post hoc comparisons indicate differences in the aforementioned constructs between the scores of the 9–10 age group and the 17–18 group (p < .001), in all cases the 9-10 age group obtained better results (lower score in depression and higher scores in appraisal and HRQoL) than the 17-18 age group.

Revista de Educación, 386. October-December 2019, pp. 181-206 191 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

Profiles Related to Bullying and Relationships Between Variables

Following the criteria described above, the participants were grouped into different profiles according to the role played in bullying (see Table 1). Of them, 50.9% is involved in bullying, with 39.6% being victims, 1.1% bullies, and 10.2% bully-victims. However, only 8.1% of the participants obtained zero in the total score of the questionnaire in the victimization dimension (having scored never in all seven items), whereas 71.9% obtained a total score equal to or greater than 3.

TABLE I. Profiles of gifted participants by their role in traditional bullying (n =285)

Total (n=285) Male (n=175) Female (n=109) Role f (%) f (%) f (%)

Victim 113(39.6) 69(39.4) 44(52.3)

Bully 3(1.1) 2(1.1) 1(0.9)

Bully-victim 29(10.2) 21(12) 7(6.4)

No problem 140(49.1) 83(47.5) 57(53.4)

Note. f = frequency; % = percentage

Table 2 shows the mean scores and standard deviations for each of the items of the victimization and aggression dimensions. The means for direct verbal-victimization, indirect verbal-victimization and direct social violence are notably higher in victims and bully-victims than in the uninvolved.

192 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

TABLE II. Mean scores and standard deviations in each of the items for the different categories and the sample as a whole

No problems Victim Bully Bully-victim Total (n=140) (n=113) (n=3) (n=29) (n=285) M(SD) M(SD) M(SD) M(SD) M(SD) Direct-physical 0.38(0.49) 1.04(0.87) 0.00(0.00) 1.55(0.98) 0.75(0.82) Indirect-physical 0.21(0.41) 0.89(0.92) 0.33(0.58) 0.76(0.69) 0.53(0.75) Direct-verbal 0.59(0.49) 1.86(0.92) 0.67(0.58) 2.29(1.28) 1.26(1.04) Indirect-verbal 0.42(0.50) 1.71(0.99) 0.33(0.58) 2.03(1.24) 1.10(1.07) Direct-social 0.42(0.50) 1.88(1.02) 0.33(0.58) 2.17(1.37) 1.18(1.14) Victimization Indirect-social 0.22(0.41) 1.43(1.06) 0.33(0.58) 1.66(1.34) 0.85(1.05) Psychological 0.20(0.40) 0.91(0.94) 0.00(0.00) 1.31(1.04) 0.59(0.84) Direct-physical 0.23(0.42) 0.32(0.47) 0.33(0.58) 0.83(0.73) 0.33(0.51) Indirect-physical 0.01(0.11) 0.07(0.26) 0.00(0.00) 0.20(0.41) 0.06(0.23) Direct-verbal 0.41(0.50) 0.54(0.50) 1.67(0.58) 1.48(0.83) 0.59(0.63) Indirect-verbal 0.34(0.48) 0.35(0.48) 1.00(1.00) 0.93(0.84) 0.41(0.56) 0.17(0.37) 0.26(0.44) 0.67(0.58) 0.93(1.07) 0.29(0.56) Aggression Direct-social Indirect-social 0.07(0.26) 0.07(0.26) 0.00(0.00) 0.10(0.31) 0.07(0.26) Psychological 0.04(0.20) 0.14(0.35) 0.00(0.00) 0.52(0.83) 0.13(0.40)

M=Mean;SD=Standard Deviation

Table III shows the different sub-roles relating to the bystander role following the categorization proposed by Salmivalli et al. (1996). There are no significant gender differences in the distributions of the sub-roles (χ2 = .953; p = .917).

TABLE III. Distribution of the observation subroles for the whole sample and by gender

Total (n=278) Male (n=173) Female (n=105) f (%) f (%) f (%) Assistant of the bully 4(1.4%) 2(1.2%) 2(1.9%) Reinforcer of the bully 2(0.7%) 1(0.6%) 1(0.9%) Bystander 40(14.4%) 27(15.6%) 13(12.4%) Pro-victim 63(22.7%) 38(22%) 25(23.8%) Defender 169(60.8%) 105(60.6%) 64(61%)

Note. f = frequency; % = percentage

Revista de Educación, 386. October-December 2019, pp. 181-206 193 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

Table IV shows the partial correlations (controlling by age) of all dimensions considered in the study. We find that victimization is significantly and negatively correlated with HRQoL r ( = -.441; p < .001) and the sense of belonging (r = -.439; p < .001), whereas is significantly and positively correlated with aggression (r = .456; p < .001) and stress (r = .543; p < .001).

TABLE IV. Partial correlations between the study dimensions controlling by age (n=278)

1 2 3 4 5 6 7 8 1.Victimization 1 2.Aggression .456** 1 3.Stress .543** .229** 1 4.Depression .463** .180* .706** 1 5.Anxiety .470** .156* .757** .784** 1 6.Appraisal -.138* -.223** -.223** -.403** -.268** 1 7.Belonging -.439** -.183** -.530** -.512** -.414** 473** 1 8.Perceived Quality -.411** -.191** -.564** -.557** -.463** .476** .563** 1 of Life

M 6.23 1.87 4.92 2.72 2.42 38.86 20.68 49.84 SD 5.28 1.84 4.67 4.07 3.53 4.94 3.49 10.46 Range 0-24 0-9 0-21 0-21 0-21 22-46 10-28 18.5-83.8

Note: M = Mean; SD = standard deviation; *p<.05; ** p < .001.

Differences in Stress, Anxiety, Depression, Quality of Life and Interpersonal Support by Role Played

Table V shows the scores for each of these variables depending on the role performed. There were significant differences in all of them (p < .001). In general, the no problems role (the student is neither victim nor bully) obtained systematically better results in terms of HRQoL, sense of belonging, depression, stress, and anxiety. The role more negatively impacted, and that consequently obtained significantly worse scores, was that of victims and bully-victims.

194 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

TABLE V. Comparisons of the total scores obtained in DASS-21 (stress, anxiety and depression), ISEL (appraisal and belonging) and KIDSCREEN-10 by role exercised in bullying by gifted students (n=285)

DASS-21 DASS-21 DASS-21 ISEL-Ap- ISEL-Belong- KID- Depression Stress Anxiety praisal ing SCREEN-10 M(SD) M(SD) M(SD) M(SD) M(SD) M(SD) (1) Victim 3.57(4.50) 6.55(5.16) 3.31(3.85) 38.38(5.22) 19.58(3.24) 47.03(9.30) (2) Bully 1.00(1.00) 2.67(3.06) 1.00(0.00) 32.00(1.00) 21.67(2.08) 53.41(6.71) (3) Bully-Victim 6.28(6.16) 8.00(5.50) 4.55(5.30) 36.48(7.13) 18.69(4.57) 44.35(10.37) (4) No problems 1.33(2.17) 3.01(3.00) 1.29(2.21) 39.89(3.77) 21.96(2.93) 53.22(10.31) F =17.10 F =20.17, F =12.20 F =6.94 F =15.30 F =11.88 Differences 3,281 3,281 3,281 3,281 3,281 3,281 p<.001 p<.001 p<.001 P<.001 p<.001 p<.001 Games-Howel 1>4; 3>4; 1>4; 3>4; 1>4; 3>4; 1>2; 2<4 1<4; 3<4; 1<4; 3<4; post hoc

Note: M=Mean; SD=Standard deviation

Discussion

This work contributes to our understanding of a reality that has been little studied up to now: bullying in gifted students. Paying attention to this group’s need and challenges should be part of an inclusive education of quality. However, there are scarce studies on this topic at the global level and none in the case of Spain. The main goal of this study was to examine the prevalence of bullying in a large group of gifted children from all over Spain, and its distribution among the different bullying- roles that can be played. It was found that up to 49.8% of the sample reported different types of victimization (i.e., victims and bully-victims). Specifically, the role of victim has a prevalence of 39.6%, 1.1% that of bullies, and 10.2% that of bully-victims. However, 71.9% of the students obtained a total score equal to or greater than 3. Prevalence data are lower than those found by Peterson and Ray (2006a, 2006b), who identified up to 67% victimization a decade ago. However, this study is not in line with that of Parker (2011), which shows that gifted students are not particularly vulnerable to bullying. Similarly, our results seem to contradict other studies that

Revista de Educación, 386. October-December 2019, pp. 181-206 195 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

did not find any significant differences between gifted students and non- gifted in relation to bullying and the type of aggression suffered (Oliveira & Barbosa, 2012; Peters & Bain, 2011). The greatest discrepancy is with the studies by Erwin (2015), and Estell et al. (2009), who suggested that gifted students suffer less victimization than their classmates, as well as with the results of Ryoo et al. (2017) who found no differences at all in the victimization of gifted students when compared with the control group (general population). Mostly, these differences may be explained by three issues: 1) the use of different tools and scaling criteria; 2) contextual differences, since most of the literature comes from the USA, with its own idiosyncrasy and culture; 3) time period over which bullying behaviors are assessed, which is larger than usual in this research (approximately 7 months, compared with one month or two months of other studies). In general, our results show that prevalence of bullying is greater in gifted children than in the general population. For instance, a recent study in a similar age range in Spain, that employs the same assessment tool, the Spanish version of the European Bullying Intervention Project Questionnaire (EBIP-Q) found a victimization rate of 11.7%, a severe victimization of 2.4%, and an aggression rate of 4% (González-Cabrera et al., 2017), percentages that are below what we have found for gifted children. The mean scores that we found for each victimization item are also greater than those indicated by Ortega et al. (2016) in another Spanish general sample study using the same assessment tool. Other Spanish studies, such that of Díaz-Aguado, Martínez & Martín, (2013) found that 3.8% of their participants were victims and that 2.4% were bullies, whereas Save the Children (2016) found a 9.3% rate of victimization, again percentages substantially lower than ours. In the international context, the Organisation for Economic Co- operation and Development (OECD) admits that bullying is a serious problem that has been worsened by the use of technology. The percentage detected in the OECD survey in the case of Spain is 6.1% (lower than the OECD mean value of 8.9%) (Ministerio de Educación, Cultura y Deporte, 2017). However, the review work by Modecki et al. (2014) places the mean rate of prevalence for aggression in traditional bullying at 35%, and for victimization at 36%. In our study, the values relating to aggression are noticeably lower and, in contrast, are considerably higher for victimization. This suggests that the prevalence of victimization among gifted students may be higher than that of the general population, and

196 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing this makes it clear that we need more in-depth research to compare gifted and non-gifted samples. Regarding the above, if we compare the prevalence of victimization obtained in this study (39.6%) with the 30% of a study in students with special educational needs (Rodriguez-Hidalgo et al., 2019) using the same questionnaire and cut-off point (EBIP-Q), the tendency shows that the samples that present singularities are more prone to victimization by their peer groups. In relation to gender, victimization and aggression scores are significantly higher for boys than for girls. This is in line with the results from the meta-analysis by Cook, Williams, Guerra, Kim and Sadek (2010), although the existence of gender of differences is still under discussion, since the effect sizes are generally low (Zych et al., 2015). Regarding age, no significant differences were found. Although the literature usually identifies the 11–14 age bracket as the most problematic (Garaigordobil et al., 2014; Pérez-Fuentes et al, 2015), there is not enough evidence and this issue continues to be discussed (Zych et al., 2015; Zych et al., 2016). To sum up, our results partially support what we hypothesized on gender and age. This might be due to the over representation of students aged 9 and 10. As we highlighted above, the disparity of results could be partly explained by methodological and contextual differences (Berne et al., 2013; Zych et al., 2016). What makes our study distinctive is that our participants share features associated with being gifted that make them a minority group with special characteristics and educational needs. In this sense, our results support a specific examination of bullying in this group, to be added to the specific support we must provide to them. A major effort in training the teaching staff and raising their awareness is vital in this regard. Our results match the few works that have focused on the bystander sub-roles (Salmivalli et al., 1996; Díaz-Aguado et al., 2013), since we also found that the defender and pro-victim sub-roles are the more prevalent ones. However, likewise Salmivalli et al. (1996), Salmivalli (1999) and González-Cabrera et al. (2017), we believed that the participants in our study overestimated their pro-victim behavior due to social desirability bias. In any case, we believe that it is important to keep working in this direction as different studies have shown that the adolescents who perceive that their peer group is involved in bullying have a higher

Revista de Educación, 386. October-December 2019, pp. 181-206 197 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

likelihood of becoming bullies themselves (Hinduja & Patchin, 2013). This also applies to gifted students and their group-class peers. In relation to health-related quality of life (HRQoL) in children and adolescents, this is defined as a state of complete physical, mental and social wellbeing. It has a multidimensional approach, covering the perceived physical, psychological, emotional, social and behavioral aspects (The KIDSCREEN Group Europe, 2006). According to the literature, being a victim of bullying can increase up to three times the chances of having a worse HRQoL than students who are not bullied (Hidalgo-Rasmussen et al., 2015). Our study matches these results, since the gifted children victims of bullying have significantly lower scores on KIDSCREEN-10 (see Table V). As found in many general population studies, victims of bullying report more stress and anxiety (Hawker & Boulton; 2000; Konishi &Hymel, 2009), as well as depression (Reijntjes, et al., 2010; Holt et al., 2015); this is in line with that we found in our study on gifted students. The results obtained for interpersonal support are especially worrying since, in line with other works (Demaray & Malecki, 2003; Holt & Espelage, 2007), we found that victims and bully-victims scored the lowest in this dimension. We also found that interpersonal support is negatively correlated with victimization and aggression, agreeing with the results of Wang, Iannotti and Nansel (2009). This would indicate that victims and bullies alike have low interpersonal support. This is especially concerning given that this variable has been suggested as a buffer for stressful events (Cohen & Wills, 1985) such as situations of victimization and depression. Consequently, these results along with what we know about chronic victimization in general population (Pouwels et al., 2016) and the stability of bullying over time (Calvete, Fernández-González, González- Cabrera & Gámez-Guadix, 2017), tell us that gifted children may be a group particularly vulnerable to prolonged victimization, with the risk that this entails for their health and wellbeing in the long term (Reijntjes et al., 2015). It also confirms another fact that we hypothesized, i.e., that victimization has a strong negative psychological impact (especially in the most severe cases). All this should be taken into account when implementing potential intervention programs. This study also has some limitations. Firstly, it could be argued that the fact that we accessed and recruited participants through gifted children associations and their legal guardians may have biased the

198 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing results, as those who have had experiences of bullying could have been particularly invited to participate. This possible bias of self-selection may be influencing the reported prevalence of school bullying byexaggerating the problem. This is why the study should be considered as exploratory. However, these initiatives are common in developing sciences, and the recruitment of difficult-to-access population subgroups by means of adaptations of the snowball sampling strategy is not uncommon (Sadler, Lee, Lim, & Fullerton, 2010). In addition, it provides an advantage, as it is a homogeneous convenience sample, which allows estimating the results with a clear generalizability towards the gifted collective (Jager, Putnick, & Bornstein, 2017). A second limitation comes from the use of self-reports with the response bias that these may entail. Thirdly, the study is cross-sectional, and the sampling, although extensive, could have benefited from having more participants. Fourthly, carrying out an online study can also create measurement problems that we may be unaware of. However, we attempted to minimize them by the protocols and control mechanisms that we implemented. It is worth noting that the procedure we followed is the only way to access a population that is geographically disperse and very difficult to locate. Finally, it is necessary to be cautious when extrapolating these results, which must be considered as a first approach to the reality of bullying in gifted students. In conclusion, this work provides empirical evidence of the high prevalence of victimization among Spanish gifted students. Victims and bully-victims present a major impact on psychological wellbeing and, particularly, on health-related quality of life, stress, anxiety, and depression. The conclusion of this study is relevant for the teaching staff that is ultimately responsible for the wellbeing of the children and should decide which actions are to be taken in order to cover their educational and emotional needs.

References

Aboujaoude, E., Savage, M. W., Starcevic, V., & Salame, W. O. (2015). Cyberbullying: review of an old problem gone viral. Journal of Adolescent Health, 57(1), 10–18. https://doi.org/10.1016/j. jadohealth.2015.04.011.

Revista de Educación, 386. October-December 2019, pp. 181-206 199 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

Albdour, M., & Krouse, H. J. (2014). Bullying and victimization among African American adolescents: A literature review. Journal of child and adolescent psychiatric nursing, 27(2), 68-82. https://doi.org/10.1111/ jcap.12066. Berne, S., Frisen, A., Schultze-Krumbholz, A., Scheithauer, H., Naruskov, K., Luik, P., et al. (2013). Cyberbullying assessment instruments: A systematic review. Aggression and Violent Behavior, 18(2), 320-334. http://doi.org/10.1016/j.avb.2012.11.022. Brighi, A., Ortega, R., Pyzalski, J., Scheithauer, H., Smith, P. K., Tsormpatzoudis, C., …Thompson, J. (2012). European Cyberbullying Intervention Project Questionnaire – ECIPQ [Unpublished questionnaire]. Retrieved from bullyingandcyber.net. Brighi, A., Ortega, R., Pyzalski, J., Scheithauer, H., Smith, P. K., Tsormpatzoudis, C., …Thompson, J. (2012). European Cyberbullying Intervention Project Questionnaire – ECIPQ [Unpublished questionnaire]. Retrieved from bullyingandcyber.net. Calvete, E., Fernández-González, L., González-Cabrera, J. M., & Gámez- Guadix, M. (2017). Continued Bullying Victimization in Adolescents: Maladaptive Schemas as a Mediational Mechanism. Journal of Youth and Adolescence, 1–11. http://doi.org/10.1007/s10964-017-0677-5. Castelló, A., & Batlle, C. (1998). Aspectos teóricos e instrumentales en la identificación del alumno superdotado y talentoso. Propuesta de un protocolo. FAISCA, 6, 26-66. Clark, B. (2012). Growing up gifted (8th Ed.). Columbus, OH: Charles E. Merrill. Cohen, S., & Wills, T. A. (1985). Stress, social support, and the buffering hypothesis. Psychological Bulletin, 98, 310–357. http://doi. org/10.1037/0033-2909.98.2.310. Cook, C. R., Williams, K. R., Guerra, N. G., Kim, T. E., & Sadek, S. (2010). Predictors of bullying and victimization in childhood and adolescence: A meta-analytic investigation. School Psychology Quarterly, 25(2), 65– 83. http://doi.org/10.1037/a0020149. Dalosto, M. D. M., & Alencar, E. M. (2013). Manifestações e prevalência de bullying entre alunos com altas habilidades/superdotação. Revista Brasileira de Educação Especial, 19(3), 363-378. Daza, P., Novy, D., Stanley, M., & Averill, P. (2002). The Depression Anxiety Stress Scale-21: Spanish translation and Validation with a Hispanic sample. Journal of Psychopathology and Behavioral Assessment, 24, 195-205. https://doi.org/10.1023/A:1016014818163.

200 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

Demaray, M. K., & Malecki, C. K. (2003). Perceptions of the frequency and importance of social support by students classified as victims, bullies, and bully/victims in an urban middle school. School Psychology Review, 32, 471–489. Díaz-Aguado, M. J., Martínez, R., & Martín, J. (2013). El acoso entre adolescentes en España: Prevalencia, papeles adoptados por todo el grupo y características a las que atribuye la victimización. Revista de Educación, 362, 348-379. https://doi.org/10-4438/1988-592X- RE-2011-362-164. Erwin, J. (2015). Prevalence and Impact of Peer Victimization Among Gifted Adolescents (Doctoral Dissertation). University of California, Berkeley, United States. Estell, D. B., Farmer, T. W., Irvin, M. J., Crowther, A., Akos, P., & Boudah, D. J. (2009). Students with exceptionalities and the peer group context of bullying and victimization in late elementary school. Journal of Child and Family Studies, 18(2), 136–150. http://doi.org/10.1007/ s10826-008-9214-1. Festl, R., Vogelgesang, J., Scharkow, M., & Quandt, T. (2017). Longitudinal patterns of involvement in cyberbullying: Results from a Latent Transition Analysis. Computers in Human Behavior, 66, 7–15. http:// doi.org/10.1016/j.chb.2016.09.027. Francis, R., Hawes, D. J., & Abbott, M. (2016). Intellectual giftedness and psychopathology in children and adolescents: A systematic literature review. Exceptional Children, 82(3), 279–302. http://doi. org/10.1177/0014402915598779. Gagné, F. (2015). From genes to talent: the DMGT/CMTD perspective. Revista de Educación, 368, 12–37. https://doi.org/10.4438/1988- 592X-RE-2015-368-289. Garaigordobil, M., & Martínez-Valderrey, V. (2014). Cyberprogram 2.0. Un Programa de intervención para la prevenir y reducir el ciberbullying. Madrid: Pirámide. Garaigordobil, M., Martínez-Valderrey, V., & Aliri, J. (2014). Victimización, percepción de la violencia y conducta social. Infancia y Aprendizaje, 37(1), 90–116. http://doi.org/10.1080/02103702.2014.881651. Gini, G., & Pozzoli, T. (2013). Bullied children and psychosomatic problems: a meta-analysis. Pediatrics, 132, 720-729. https://doi. org/10.1542/peds.2013-0614.

Revista de Educación, 386. October-December 2019, pp. 181-206 201 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

González-Cabrera, J., Balea, A., Vallina, M., Moya, A., & Laviana, O. (2017). Informe ejecutivo del Proyecto CIBERASTUR [CYBERASTUR project executive report]. Consejería de Educación y Cultura del Principado de Asturias. https://goo.gl/wLxQ6i. Harris, J. R. (1995). Where is the child’s environment? A group socialization theory of development. Psychological Review, 102(3), 458–489. http:// doi.org/10.1037/0033-295X.102.3.458. Haslam, S., Ellemers, N., Reicher, S. D., Reynolds, K., & Schmitt, M. (2010). The social identity perspective today: An overview of its defining ideas. Rediscovering Social Identity. In T. Postmes, & N. R. Branscombe (Eds.), Rediscovering Social Identity: Core Sources (pp. 341-356). New York: Psychology Press. Hawker, D. S. J., & Boulton, M. J. (2000). Twenty Years’ Research on Peer Victimization and Psychosocial Maladjustment: A Meta-analytic Review of Cross-sectional Studies. Journal of Child Psychology and Psychiatry, 41(4), 441–455. http://doi.org/10.1111/1469-7610.00629 Hidalgo-Rasmussen, C., Molina, T., Molina, R., Sepúlveda, R., Martínez, V., Montaño, R., Gonález, E., & George, M. (2015). Influence of bullying on the quality of life perception of Chilean students. Revista Médica de Chile, 143(6), 716-723. Hinduja, S., & Patchin, J. W. (2013). Social influences on cyberbullying behaviors among middle and high school students. Journal of youth and adolescence, 42(5), 711-722. https://doi.org/10.1007/s10964-012- 9902-4 Holt, M. K., & Espelage, D. L. (2007). Perceived social support among bullies, victims, and bully-victims. Journal of Youth and Adolescence, 36(8), 984–994. http://doi.org/10.1007/s10964-006-9153-3. Holt, M. K., Vivolo-Kantor, A. M., Polanin, J. R., Holland, K. M., DeGue, S., Matjasko, J. L., … Reid, G. (2015). Bullying and Suicidal Ideation and Behaviors: A Meta-Analysis. Pediatrics, 135(2), e496–e509. –1864http://doi.org/10.1542/peds.2014. Hymel, S., McClure, R., Miller, M., Shumka, E., & Trach, J. (2015). Addressing school bullying: Insights from theories of group processes. Journal of Applied Developmental Psychology, 37(1), 16–24. http://doi. org/10.1016/j.appdev.2014.11.008. Jager, J., Putnick, D. L., & Bornstein, M. H. (2017). II. More than just convenient: The scientific merits of homogeneous convenience samples. Monographs of the Society for Research in Child Development, 82(2), 13-30. https://doi.org/10.1111 / mono.12296.

202 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

Konishi, C., & Hymel, S. (2009). Bullying and Stress in Early Adolescence. Journal of Early Adolescence, 29(3), 333–356. http://doi. org/10.1177/0272431608320126. Margolin, L. (2018). Gifted Education and the Matthew Effect. In A. Böker and K. Horvath (Eds.), Begabung und Gesellschaft (pp. 165–182). Springer Fachmedien Wiesbaden: Wiesbaden, Germany. Matthews, M. S., Ritchotte, J. A., & Jolly, J. (2014). What’s wrong with giftedness? Parents’ perceptions of the gifted label. International Studies in Sociology of Education, 24(4), 372– 393. doi:10.1080/0962 0214.2014.990225. Ministerio de Educación, Cultura y Deporte (2017). Bienestar de los estudiantes: resultados de PISA 2015. https://goo.gl/Egq2pU. Modecki, K.L., Minchin, J., Harbaugh, A.G., Guerra, N.G., & Runions, K.C. (2014). Bullying prevalence across contexts: A meta-analysis measuring cyber and traditional bullying. Journal of Adolescent Health, 55, 602– 611. http://doi.org/10.1016/j.jadohealth.2014.06.007. Nakamoto, J., & Schwartz, D. (2009). Is peer victimization associated with academic achievement? A Meta-analytic Review. Social Development, 19, 221–242. https://doi.org/10.1111/j.1467-9507.2009.00539.x. Oliveira, J. C., & Barbosa, A. J. G. (2012). Bullying among students with and without characteristics of giftedness. Psicologia: Reflexão e Crítica, (4), 25 747–755. https://doi.org/10.1590/S0102- 79722012000400014. Olszewski-Kubilius, P., Subotnik, R. F., & Worrell, F. C. (2015). Re- pensando las altas capacidades: una aproximación evolutiva. Revista de Educación, 368, 40-65. Olweus, D. (1999). Norway. In P. K. Smith, Y. Morita, J., Junger-Task, D. Olweus, R. Catalano, & P. Slee (Eds.) The nature of school bullying: A cross-national perspective (pp. 28–48). London, England: Routledge. Oostenbroek, J., & Over, H. (2015). Young children contrast their behavior to that of out-group members. Journal of Experimental Child Psychology, 139, 234–241. http://doi.org/ 10.1016/j.jecp.2015.05.009. Ortega-Ruiz, R., Del Rey, R., & Casas, J. (2016). Evaluar el bullying y el cyberbullying validación española del EBIP-Q y del ECIP-Q. Psicología de la Educación, 22, 71-79. https://doi.org/10.1016/j.pse.2016.01.004. Parker, P. (2011). Bullying and victimization rates among gifted and high- achieving students. Journal for the Education of the Gifted, 34(4), 624–643. https://doi.org/10.1177/016235321103400405.

Revista de Educación, 386. October-December 2019, pp. 181-206 203 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

Pelchar, T. K., & Bain, S. K. (2014). Bullying and victimization among gifted children in school-level transitions. Journal for the Education of the Gifted, 37, 319–336. http://doi.org/10.1177/ 0162353214552566. Pérez-Fuentes, M. del C., Gázquez, J. J., Molero, M. del M., Soler, J., & Barragán, A. B. (2015). Valores interpersonales relacionados con el perfil de agresor y víctima en adolescentes. International Journal of Psychology and Psychological Therapy, 15(2), 229–239. Peters, M. P., & Bain, S. K. (2011). Bullying and victimization rates among gifted and high achieving students. Journal for the Education of the Gifted, 34(4), 624–643. https://doi.org/10.1177/016235321103400405. Peterson, J. S. (2009). Myth 17: Gifted and talented individuals do not have unique social and emotional needs. Gifted Child Quarterly, 53, 280–282. http://doi.org/10.1177/0016986209346946. Peterson, J. S., & Ray, K. E. (2006a). Bullying and the gifted: Victims, perpetrators, prevalence, and effects. Gifted Child Quarterly, 50(2), 148–168. https://doi.org/10.1177/001698620605000305. Peterson, J. S., & Ray, K. E. (2006b). Bullying among the gifted: The subjective experience. Gifted Child Quarterly, 50(3), 252–269. https:// doi.org/10.1177/001698620605000305. Pfeiffer, S. (2018). Giftedness and talent development in children and youth. In Paul Ward, Jan Maarten Schraagen, Julie Gore, & Emilie M. Roth (Eds.),The Oxford handbook of expertise. https://doi.org/10.1093/ oxfordhb/9780198795872.013.5. Pouwels, J. L., Souren, P. M., Lansu, T. A. M., & Cillessen, A. H. N. (2016). Stability of peer victimization: A meta-analysis of longitudinal research. Developmental Review, 40, 1–24. https://doi.org/10.1016/j. dr.2016.01.001. Reijntjes, A., Kamphuis, J. H., Prinzie, P., & Telch, M. J. (2010). Peer victimization and internalizing problems in children: A meta-analysis of longitudinal studies. Child Abuse and Neglect, 34(4), 244–252. http://doi.org/10.1016/j.chiabu.2009.07.009. Reijntjes, A., Thomaes, S., Kamphuis, J. H., Bushman, B. J., Reitz, E., & Telch, M. J. (2013). Youths’ displaced aggression against in- and out- group peers: An experimental examination. Journal of Experimental Child Psychology, 115(1), 180–187. https://doi.org/10.1016/j. jecp.2012.11.010. Renzulli, J. S, & Reis, S. M. (2018). The three-ring conception of giftedness: A developmental approach for promoting creative productivity in

204 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

young people. In Pfeiffer, S. I., Shaunessy-Dedrick, E., & Foley-Nicpon, M. (Eds.), APA handbooks in psychology. APA handbook of giftedness and talent. Washington, D.C.: American Psychological Association. Rodríguez-Hidalgo, A. J., Alcívar, A., & Herrera-López, M. (2019). Traditional bullying and discriminatory bullying around special educational needs: Psychometric properties of two instruments to measure it. International Journal of Environmental Research and Public Health, 16(1), 142. https://doi.org/10.3390/ijerph16010142. Ryoo, J. H., Wang, C., Swearer, S. M., & Park, S. (2017). Investigation of transitions in bullying/victimization statuses of gifted and general education students. Exceptional Children, 83(4), 396–411. http://doi. org/10.1177/0014402917698500. Sadler GR, Lee H, Lim RS, Fullerton J. (2010). Recruiting hard-to-reach United States population sub-groups via adaptations of snowball sampling strategy. Nursing & Health Sciences, 12 (3), 369-374. https:// doi.org/10.1111/j.1442-2018.2010.00541.x. Salmivalli, C. (1999). Participant role approach to school bullying: Implications for interventions. Journal of Adolescence, 22(4), 453–459. http://doi.org/10.1006/jado.1999.0239. Salmivalli, C., Lagerspetz, K., Björkqvist, K., Österman, K., & Kaukiainen, A. (1996). Bullying as a group process: Participant roles and their relations to social status within the group. Aggressive Behavior, 22(1), 1–15. Save the Children (2016). Yo a eso no juego. Bullying y ciberbullying. Retrieved from https://www.savethechildren.es/sites/default/files/ imce/docs/yo_a_eso_no_juego.pdf. Silverman, L. K. (2002). Asynchronous development. In M. Neihart, S. M. Reis, N. M. Robinson, & S. M. Moon (Eds.), The social and emotional development of gifted children: What do we know? (pp. 31–37). Waco, TX: Prufrock Press. Smith, B., Dempsey, A., Jackson, S., Olenchak, R., & Gaa, J. (2012). Cyberbullying among Gifted Children. Gifted Education International, 28, 112–126. https://doi.org/10.1177/0261429411427652. Tajfel, H. (1982). Social Psychology of Intergroup relations. Annual Review of Psychology, 26(2000), 1–39. http://doi.org/ 10.1146/annurev. ps.33.020182.000245. The KIDSCREEN Group Europe (2006). The KIDSCREEN Questionnaires. Quality of life questionnaires for children and adolescents. Lengerich, Germany: Pabst Science Publishers.

Revista de Educación, 386. October-December 2019, pp. 181-206 205 Received: 21-09-2018 Accepted: 24-07-2019 González-Cabrera, J., Tourón, J., Machimbarrena, J.M., León-Mejía, A. Gutiérrez-Ortega, M. Exploratory study on bullying in gifted students: prevalence and psychological wellbeing

Tourón, J. & Tourón, M. (2016). Identification of verbal and mathematical talent: The relevance of ‘out of level’ measurement. Anales de Psicología, 32(3), 638–651. https://doi.org/10.6018/analesps.32.3.259401. Trujillo, H., Martos, F., & González-Cabrera, J. (2012). Adaptación al español del “Interpersonal Support Evaluation List” (ISEL). Universitas Psychologica, 3, 969–978. Wang, J., Iannotti, R. J., & Nansel, T. R. (2009). School Bullying Among Adolescents in the United States: Physical, Verbal, Relational, and Cyber. Journal of Adolescent Health, 45(4), 368–375. http://doi. org/10.1016/j.jadohealth.2009.03.021. Yuste, C. (1989). Batería de aptitudes diferenciales y generales. BADYG-S. Madrid: Manual técnico. Madrid: Ed. CEPE. Zych, I., Ortega-Ruiz, R., & Del Rey, R. (2015). Systematic review of theoretical studies on bullying and cyberbullying: Facts, knowledge, prevention, and intervention. Aggression and Violent Behavior, 23, 1–21. https://doi.org/10.1016/j.avb.2015.10.001. Zych, I., Ortega-Ruiz, R., & Marín-López, I. (2016). Cyberbullying: a systematic review of research, its prevalence and assessment issues in Spanish studies. Psicología Educativa, 22, 5-18. https://doi. org/10.1016/j.pse.2016.03.002.

Contact address: Joaquín González-Cabrera. Universidad Internacional de La Rioja. Facultad de Educación. Av. de la Paz, 137 26006, Logroño (La Rioja). España. E-mail: [email protected].

206 Revista de Educación, 386. October-December 2019, pp. 181-206 Received: 21-09-2018 Accepted: 24-07-2019 Reviews

Reviews

González-Falcón, I. (Ed) (2018). Understanding Cultural Diversity in Education. Perceptions, Opportunities and Challenges. New York: NOVA. Pp. 310. ISBN: 978-1-53614-061-3

In the multicultural world we live in today, we need to resort to self- understanding, recognition and self-criticism of societies and their individuals; societies where space-time borders are sometimes blurred, while in others they are defined in order to impede migratory movements, an effort that seems futile. The reviewed work, published in November 2018, assumes cultural diversity as wealth and contributes theoretical research and applied works researching the guiding conceptions of teachers and different educational professionals regarding this notion. The volume, consisting of ten chapters, is the result of a collective work by European, American and Asian experts and offers an interdisciplinary approach with contributions from anthropology, sociology and pedagogy. In the first chapter, Vincenzo Matera (University of Bologna) reflects upon the concepts of culture and cultural diversity, establishing space and time as variables in them. From this perspective, human beings would be a mixture of tradition and innovation, of past and future, and would need to understand diversity in order to understand themselves and their society. The second chapter, by Inmaculada González-Falcón; María de la O Toscano-Cruz and Inmaculada Gómez-Hurtado (University of Huelva), studies the processes of migration and reconstruction of cultural identity, analysing the economy, society and education among the variables, proposing the maintenance of social cohesion and improvement of socio-educational inclusion processes as goals, in addition to reforms in the financial system. In the third chapter, by Lina Higueras-Rodríguez and Marisa Mártires (University of Granada and Visual Arts Department of Olhao Art School,

Revista de Educación, 386. October-December 2019, pp. 207-211 209 Portugal), the authors analyse the most suitable educational models for cultural diversity management, while also highlighting the role of the Nation-State and societies in assessing and responding to diversity, as well as the potential of the inclusive model. Next, a chapter addressed by Dr. Moshe Tatar (Hebrew University of Jerusalem) presents the result of an analysis of research into the way cultural diversity is perceived and managed in more than a hundred Primary and Secondary schools in Israel. The fifth chapter, by Paola Dusi (University of Verona), offers a reflection on the role played by school in our current multicultural society and examines the intercultural skills teachers should have. In chapter six, Asunción Moya and Mª Pilar García Rodríguez (University of Huelva) analyse the shortcomings of teaching staff in terms of cultural diversity management training and provide a formative guide in intercultural education that has garnered several international awards, thanks to its ability to respond to the needs detected. Chapter seven, by Eric D. Torres, Martín Valdiviezo and Carmen Días- Bazoindagan, includes the results of joint research carried out by the Wisconsin-Eau Claire University and the Pontifical Catholic University of Peru. The curricular schedules reveal how, despite the difficulty due to the long history of discrimination in a country like Peru, there are indications of a new cultural identity that is more plural, diverse and respectful. The eighth chapter, created by María José Arroyo and Ignacio Berzosa (University of Valladolid), analyses the role of second language learning for foreign students in terms of accessing the school curriculum and achieving their academic milestones. In chapter nine, Stefania Pontrandolfo (University of Verona) carries out research on the situation of the Roma community in Europe and particularly in Italy, where the rejection of this ethnic group in educational institutions is reflected. The book closes with a chapter by researcher Jong Youl Hong (Hankuk University of Foreign Studies of Seoul), which provides information on two European good practice projects, “Intercultural City “and” Creative Europe”, which promote interculturality in urban environments. In short, the work allows reflection on the most salient conceptions, opportunities and challenges for the positive management of cultural diversity in school and beyond. The work is indubitably a crucial resource

210 Revista de Educación, 386. October-December 2019, pp. 207-211 for all those interested in intervention and research on cultural diversity in the socio-educational scope.

Cristina Goenechea Permisán

Revista de Educación, 386. October-December 2019, pp. 207-211 211