revista de eDUCACIÓN Nº 381 JULY-SEPTEMBER 2018 revista de eDUCACIÓN Nº 381 JULY-SEPTEMBER 2018 Nº 381 July-September 2018 Quarterly Journal Starting year: 1952
MINISTERIO DE EDUCACIÓN, CULTURA Y DEPORTE SECRETARÍA DE ESTADO DE EDUCACIÓN, FORMACIÓN PROFESIONAL Y UNIVERSIDADES
Instituto Nacional de Evaluación Educativa Ministerio de Educación, Cultura y Deporte Paseo del Prado, 28, 4.ª planta 28014 Madrid España
Edita © SECRETARÍA GENERAL TÉCNICA Subdirección General de Documentación y Publicaciones.
Catálogo de publicaciones del Ministerio: mecd.gob.es Catálogo general de publicaciones oficiales: publicacionesoficiales.boe.es
Edición: 2018 NIPO línea: 030-15-019-6 NIPO ibd: 030-15-018-0 ISSN línea:1988-592X 0034-8082 ISSN papel: 0034-8082 Depósito Legal: M.57/1958
Diseño de la portada: Dinarte S.L. Maqueta: Solana e hijos, Artes Gráficas S.A.U. MANAGING BOARD EDITORIAL TEAM
CHAIR Colaboradores externos: Jorge Mañana Rodríguez and Jesús García Marcial Marín Hellín Laborda Secretario de Estado de Educación, Formación Profesional y Universidades
MEMBERS SCIENTIFIC ADVISERS Jorge Sainz González Secretario General de Universidades International
Marco A. Rando Rando Aaron Benavot (State University of New York SUNY-Albany);Abdeljalil Akkari (Profesor de la Universidad de Ginebra, Suiza); Jorge Baxter Director General de Evaluación y Cooperación Territorial (Organización de Estados Americanos); Mark Bray (University of Hong Kong); José Joaquín Brunner (Universidad Diego Portales, Chile);Andy Rosalía Serrano Velasco Hargreaves (Lynch School of Education. Boston College. Editor-in-Chief Directora General de Formación Profesional of the Journal of Educational Change); Seamus Hegarty (President, International Association for the Evaluation of Educational Achievement, ——— IEA); Felipe Martínez Rizo (Profesor del Departamento de Educación. Director General de Política Universitaria Universidad Autónoma de Aguascalientes, México); Jaap Scheerens (University of Twente, Netherlands. INES Project, OCDE); Andreas Mónica Fernández Muñoz Schleicher (Head of the Indicators and Analysis Division, Directory for Secretaría General Técnica Education, OCDE).
Carmen Tovar Sánchez Directora del Instituto Nacional de Evaluación Educativa National
Amparo Barbolla Granda Teresa Aguado (UNED); Sagrario Avezuela Sánchez (IES Lázaro Cárdenas, Subdirectora General de Documentación y Publicaciones CAM); Margarita Bartolomé (U. de Barcelona); Jesús Beltrán Llera (U. Complutense); Antonio Bolívar (U. de Granada); Josefina Cambra (Colegios Violeta Miguel Pérez de Doctores y Licenciados); Anna Camps (U. Autónoma de Barcelona); Directora del Centro Nacional de Innovación e Investigación Educativa Colectivo Ioé (Madrid); César Coll (U. de Barcelona); Agustín Dosil (U. de Santiago); Gerardo Echeita (U. Autónoma de Madrid); Juan Manuel Escudero (U. de Murcia); Mariano Fernández Enguita (U. Complutense de EDITORIAL BOARD Madrid); Joaquín Gairín (U. Autónoma de Barcelona); M.ª Ángeles Galino; J. L. García Garrido (UNED); Daniel Gil (U. de Valencia); José Gimeno CHAIR Sacristán (U. de Valencia); Fuensanta Hernández Pina (U. de Murcia); Carmen Tovar Sánchez Carmen Labrador (U. Complutense); Ramón L. Facal (IES Pontepedriña, Santiago de Compostela. Revista Iber); Miguel López Melero (U. de Málaga); Carmen Maestro Martín (IES Gran Capitán, CAM); Carlos Marcelo (U. de MANAGING EDITOR Sevilla); Elena Martín (U. Autónoma de Madrid); Miquel Martínez (U. de José Luis Gaviria Soto Barcelona); Rosario Martínez Arias (U. Complutense); Mario de Miguel (U. de Oviedo); Inés Miret (Neturity, Madrid); Juan Manuel Moreno Olmedilla ASSISTANT EDITOR (Banco Mundial); Gerardo Muñoz (Inspección de Madrid); Gema Paniagua David Reyero García (E. Atención Temprana, Leganés); Emilio Pedrinaci (IES El Majuelo, Sevilla); Ramón Pérez Juste (UNED); Gloria Pérez Serrano (UNED); Ignacio Pozo MEMBERS (U. Autónoma de Madrid); M.ª Dolores de Prada (Inspección); Joaquim Antonio Cabrales Goitia (U. Carlos III de Madrid); Caterina Casalmiglia Prats (U. de Barcelona); Manuel de Puelles (UNED); Tomás Recio (U. (U. Autónoma de Barcelona); José Luis García Garrido (U. Nacional de de Cantabria); Luis Rico (U. de Granada); Enrique Roca Cobo; Juana M.ª Educación a Distancia); Antonio Lafuente García (CSIC); Leoncio López Sancho (U. de Barcelona); Juan Carlos Tedesco (UNESCO); Alejandro Ocón (CSIC) ; Clara Eugenia Núñez Romero (U. Nacional de Educación a Tiana Ferrer (UNED); Consuelo Uceda (Colegio La Navata, Madrid); Distancia); Arturo de la Orden Hoz (U. Complutense de Madrid); Consuelo Vélaz de Medrano (UNED); Mercedes Vico (U. de Málaga); Lucrecia Santibáñez (RAND Corporation); Javier Tourón Figueroa Florencio Villarroya (IES Miguel Catalán, Zaragoza. Revista Suma); Antonio (U. de Navarra); Pablo Zoido (OCDE) Viñao (U. de Murcia). Presentation
Revista de Educación is a scientific journal published by the Ministerio de Educación, Cultura y Deporte. Founded in 1940, and since 1952 called Revista de Educación, it has been a privileged witness of the development of education in the last decades, and an acknowledged means for the dissemination of education research and innovation, both from a national and international perspectives. It is currently assigned to the Instituto Nacional de Evaluación Educativa within the Dirección General de Evaluación y Cooperación Territorial and it is published by the Subdirección General de Documentación y Publicaciones of the Ministerio de Educación, Cultura y Deporte.
Each year we publish four issues. Starting next issue (No. 361), the magazine will have three sections: Research, Essays and Education Experiences, all of them submitted to referees. In the first issue of the year there is also an indexof bibliography, and in the second number a report with statistic information about the journal process of this period and the impact factors, as well as a list of our external advisors.
From 2006 to the second number of 2012 (May-August 358), Revista de Educación was published in a double format, paper and electronic. The paper edition included all the articles in the especial section, the abstracts of articles pertaining to the rest of sections, and an index of reviewed and received books. The electronic edition contains all articles and reviews of each issue, and it is available through this web page (www.mecd.gob.es/revista-de-educacion/), where it is possible to find more interesting information about the journal. From the 358 number Revista de Educación becomes exclusively an online publication.
Revista de Educación assesses, selects and publishes studies framed in well established lines of research, mainly: methodologies of education investigation and assessment; analysis of education systems and public policies; evolution and history of contemporary education systems; education reforms and innovations; quality and equity in education; curriculum; didactics; school organization and management; attention to diversity and inclusive education; educational guidance and tutorship; teacher selection, training and professional development; international cooperation for the development of education.
5 Revista de Educación is available through the following data bases:
■ National databases: isoc, beg (gencat), psicodoc, dialnet, y redined (Red de Bases de Datos de Información Educativa).
■ International databases: Social Sciences Citation Index® (ssci), Social Scisearch®, scopus, Sociological Abstracts (csa Illumina), pio (Periodical Index Online, Reino Unido), iresie (México), icist (Canadá), hedbib (International Association of Universities - Unesco International Bibliographic Database on Higher Education), swetsnet (Holanda).
■ Journal evaluation systems: Journal Citation Reports/Social Sciences Edition (jcr), European Reference Index for the Humanities (erih), Latindex (Iberoamericana), scimago Journal & Country Rank (sjr), resh, Difusión y Calidad Editorial de las Revistas Españolas de Humanidades y Ciencias Sociales y Jurídicas (dice), carhus plus+, Matriu d’Informació per a l’Avaluació de Revistes (miar), Clasificación Integrada de Revistas Científicas circ( ).
■ Directories: Ulrich’s Periodicals Directory.
■ National catalogues: Consejo Superior de Investigaciones Científicas (csic-isoc), Red de Bibliotecas Universitarias (rebiun), Centro Nacional de Innovación e Investigación Educativa (Ministerio de Educación, Cultura y Deporte), Catálogo Colectivo de Publicaciones Periódicas en Bibliotecas Españolas (Ministerio de Educación, Cultura y Deporte).
■ International catalogues: WorldCat (usa), Online Computer Library Center (usa), Library of Congress (lc), The British Library Current Serials Received, King’s College London, Catalogue Collectif de France (CCFr), Centro de Recursos Documentales e Informáticos de la Organización de Estados Iberoamericanos (oei), copac National, Academic and Specialist Library Catalogue (United Kingdom), sudoc Catalogue du Système Universitaire de Documentation (France), zdb Zeitschriftendatenbank (Alemania).
Revista de Educación does not necessarily agree with opinions and judgements maintained by authors
6 Index
Research
Claudia-Elena Velázquez-Olmedo, Leticia Gallegos-Cázares & Elena Calderón-Canales: Construction of intuitive representations about sound in preschool children...... 11
Caroline Shackleton: Linking the University of Granada CertAcles listening test to the CEFR...... 35
Franciso Javier Murillo & Cynthia Martínez-Garrido: Impact of the economic crisis on school segregation in Spain...... 61
María Luz Diago Egaña, María José Cuetos Revuelta & Patricia González González: Analysis of the Learning Styles measurement tools...... 89
Josetxu Orrantia, Sara San Romualdo, Rosario Sánchez, Laura Matilla, David Muñez & Lieven Verschaffel: Numerical magnitude processing and mathematics achievement...... 127
Patricia Suárez & Trinidad Suárez: Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)...... 147
Ximena Dueñas Herrera, Ana Bolena Escobar Escobar, Silvana Godoy Mateus & Jorge Leonardo Duarte Rodríguez: Linking coaches and student achievement: PTA in Colombia...... 173
Ángeles Blanco-Blanco: State of scientific practices and educational research. Potential challenges for the next decade...... 199
Susana López Solé, Mireia Civís Zaragoza & Jordi Díaz-Gibson: Social Capital and Social Networks of Teachers: Systematic Review...... 223
7 David Aguilera Morales, Tobías Martín Páez, Víctor Valdivia Rodríguez, Ángela Ruiz Delgado, Leticia Williams Pinto, José Miguel Vílchez González & Francisco Javier Perales Palacios: Inquiry-based Science Education. A systematic review of Spanish production...... 249
Raquel Fernández-Cézar, Natalia Pinto Solano & Marta Muñoz Hernández: Do experimentation outreach programs improve attitudes towards school science?...... 275
Reviews...... 297
8 Research
Construction of intuitive representations about sound in preschool children1
Formación de representaciones intuitivas acerca del sonido en niños de preescolar
DOI: 10.4438/1988-592X-RE-2017-381-379
Claudia-Elena Velázquez-Olmedo Leticia Gallegos-Cázares Elena Calderón-Canales Universidad Nacional Autónoma de México
Abstract The present research aimed to know the intuitive representations that preschool children built about sound. The study is based on representational theory, which studies the construction of scientific knowledge through the development of mental representations, understanding by representation all construction, cognitively mediated by the subject, which allows him to account for a phenomenon or situation, and the theory of corporal cognition (embodiment), which refers to the role of the body in the construction and explanation of these representations. A case study was carried out consisting of six children of 3rd grade of preschool. The data were obtained through the individual application of a semi-structured interview based on the themes: production, perception and propagation of sound, which involved the use of graphic and material which allowed contextualizing and delimiting the interview. All the interviews were recorded in audio and video. Two different analyses were carried out in parallel, in the first, analysis categories were constructed from the verbal responses of the children. In the second analysis, the gestures used by the children to explain their answers were identified and categorized, or in some cases the gesture itself was the response. The results reveal that children have a substantialist representation
(1) This ressearch was developed with the support of Secretaría de Educación Pública y el Consejo Nacional de Ciencia y Tecnología -México-(Ref. nº 240419. SEP-CONACyT). Acknowledgements: Universidad Nacional Autónoma de México (UNAM) and Instituto de Ciencias Aplicadas y Tecnología (ICAT).
Revista de Educación, 381. July-September 2018, pp. 11-34 11 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
of sound, that is, they consider that sound has similar properties to those objects (mass and defined trajectory), based on a materialistic intuition about sound, which responds to experiences phenomenologically guided by perception. Some children finally infer that there is a relationship between vibrations and sound but they are not able to make explicit what kind of relationship it is.
Keywords: Preschool education, Science education, mental representations.
Resumen La presente investigación tuvo como objetivo conocer las representaciones intuitivas que tienen los niños de nivel preescolar acerca del sonido. El estudio se fundamenta en la teoría representacional, la cual estudia la construcción de conocimiento científico mediante el desarrollo de representaciones mentales, entendiendo por representación toda construcción, mediada cognitivamente por el sujeto, que le permite dar cuenta de un fenómeno o situación, y la teoría de la cognición corporal (embodiment), la cual hace referencia al papel del cuerpo en la construcción y explicitación de estas representaciones. Se realizó un estudio de casos constituido por seis niños de 3er grado de preescolar, los datos se obtuvieron mediante la aplicación individual de una entrevista semiestructurada basada en los temas: producción, percepción y propagación del sonido, que implicó el uso de material gráfico e instrumental lo que permitió contextualizar y delimitar las entrevistas, registradas en audio y video. Se realizaron paralelamente dos análisis diferentes, en el primero se construyeron categorías de análisis a partir de las respuestas verbales de los niños, en el segundo se identificaron y categorizan los gestos utilizados por los niños para explicar sus respuestas o en algunos casos siendo el gesto la propia respuesta. Los resultados revelan que los niños tienen una representación sustancialista del sonido, es decir, consideran que el sonido tiene propiedades similares a las de los objetos (masa y trayectoria definida), basada en una intuición materialista acerca del sonido, que responde a experiencias fenomenológicas guiadas esencialmente por la percepción. Finalmente intuyen que existe una relación entre las vibraciones y el sonido pero no son capaces de hacer explícito de qué tipo de relación se trata.
Palabras clave: Educación de la primera infancia, Enseñanza de las ciencias, Cognición, Representación mental.
Introduction
Representational training in knowledge construction has been studied by several researchers (Cook, 2006, Mazens and Lautrey, 2003, Osborne and Freyberg, 1998, Postigo and López-Manjón, 2012, Pozo and Gómez-
12 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
Crespo, 1998). Although all knowledge is a representational construct, we assume that certain representations influence the formation of other types of representation during the development of scientific literacy. In this study, we use the concept of intuitive representations to refer to the set of ideas that individuals employ to interpret natural phenomena. Such representation may contradict what is established in the theories, principles and laws of scientific knowledge. We consider such representations to reflect the subject’s perceptual and cognitive structure and to be based on daily experiences, both physical and social. Ultimately, such representations create an empirical understanding of science (Moreira and Greca, 2005; Pozo and Gómez-Crespo, 1998). Regardless of school level, all students enter the classroom with intuitive representations that often are strong and persistent. Because representations of the world determine how individuals think and act, understanding how intuitive representations are formed is an important task. In the study, the intuitive representations of preschool children regarding sound are analysed. The following sections present a description of the representations, a review of previous studies on the topic and the interpretative framework of the investigation. Subsequently, the study’s results and conclusions are presented.
How do children develop their knowledge of the world?
Understanding the learning process has occupied researchers in various areas, from neuroscience to the arts and humanities. Thus, different approaches have been developed to explain learning. Although the study of how students learn sciences has a long tradition, investigations that address how children of preschool age learn science commenced only a few years ago. To develop an account of how the world works, children establish relationships between things and facts. These relationships help them explain natural phenomena, physical relationships between objects, and the role of others and themselves. This articulated set of relationships has been termed representations. These representations enable us to explain what occurs, make sense of phenomena and anticipate what will occur (Chi and Roscoe, 2002, Duschl and Grandy 2008, Nersessian,
Revista de Educación, 381. July-September 2018, pp. 11-34 13 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
1999, Tytler and Prain, 2010). For example, a visual-spatial representation enables us to situate what occurs in a location and establish relationships of dependency between events. However, it is temporal representation that gives meaning to causality. An adequate representation, which offers the possibility of experimenting more effectively (not necessarily with objects), opens up more ways for individuals to represent the world. However, having a representation of the world is not enough. We must make the world explicit to ourselves and to others in a way that generates new meanings. In addition to the explanatory function, representations seek to satisfy other needs of the subject. Such needs are not necessarily cognitive. They also possess ideological, motivational or affective aspects. Moreover, they are not fully articulated and available at any time. That is, at the moment a representation is required, the subject combines different elements from among those that are available to respond to the new representational demand. The elements of construction are always available. However, the articulation within the structure that produces meaning only appears to respond to a need at a certain time (Tytler and Prain, 2010). The demand can be to solve a practical problem or to explain a phenomenon. The explanatory function of a representation appears each time the subject must solve a problem. Thus, a representation is linked to the subject’s purpose. Importantly, the representations that each individual must form to function in the world are prior to scientific knowledge and developed before science emerged (Delval, 2007). One problem we face in the study of representations is that they cannot be assessed directly. We must infer them from what subjects do or state regarding objects or phenomena (Driver, 1989; Nersessian, 1999; Tytler and Prain, 2010). Thus, language and the ability to use abstract systems of representation substantially increase the possibilities of explanation. However, these abstract systems are only the vehicle through which the representations are made manifest.
Role of the body in representational construction
A number of researchers have adopted the concept of body cognition embodiment (Pozo, 2003) as an interpretive framework to understand and explain the representations that individuals have regarding scientific
14 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children or physical concepts (Alibali and Nathan, 2012; Calvo and Gomila, 2008 Edwards, 2009; Gibbs, 2005; Hostetter and Alibali, 2008). This concept can be considered to develop from the psychology of Jean Piaget, who claimed that cognitive abilities are based on sensorimotor skills (Wilson, 2002). Thus, Piaget believed that cognitive processes are strongly rooted in sensory-motor interactions of the body with the world. What function or functions does the body have in the cognitive process? According to Clark and Chalmers (1998) and Wilson (2002), the body has at least two distinct but related functions in the cognitive process. The first is that the body can function as acognitive limitation. In this sense, speaking or thinking about objects, whether concrete or abstract, involves the invocation, deployment or reactivation of specific patterns of body activity (i.e., the body shapes the nature of cognitive activity and the content of processed representations). Examples include a) the perception of colour (the concepts and experiences of the colours are due to the properties of the cells of the retina and the characteristics of the visual apparatus; thus, we only perceive the spectrum of visible light); b) the detection of sound, which owes its peculiarity to the distance between the ears; and c) the spatial metaphor, which is the way we conceptualize the body based on “embodied” experiences. For example, the conceptual metaphor that time is a movable object expresses an action and that “Monday comes before Tuesday” refers to movement through time of “what comes before” and “the day left behind” (Thompson et al., 1992). The second function concerns the different ways in which the body acts as a distributor for cognitive processing, i.e., operates as a system of partial interpretation of cognition. That is, as part of the cognitive process, the body “extends” into cognitive tasks within structures that are not limited to the central nervous system (non-neuronal), thus functioning as a partial performer of mental processes. Examples of non- neural processes primarily include gestures and postures, which have been addressed in studies on cognitive operations, language production, cortical plasticity and the acquisition of motor skills of the hand (Rowe and Goldin-Meadow, 2009; Rowe et al., 2008). Similarly, other research suggests that the increase in volume of movement representations is modelled by embodied experiences and that changes induced by the body help regulate the brain with respect to information processing and cognitive development (Hlustìk et al., 2004). These research outcomes suggest that the body plays an important role in representational formation.
Revista de Educación, 381. July-September 2018, pp. 11-34 15 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
Thus, purely physical activity, specifically gestures, has been considered a form of communication. In fact, for certain authors, movements play a cognitive role in the development of vocabulary and language (Rowe and Goldin-Meadow, 2009; Rowe et al., 2008). In addition, physical movement is considered to contribute to understanding meaning and to forming representations. One of the most frequently cited studies on the function of gestures in representation formation was by Corballis (2003), who notes that this process involves neurons that transmit information regarding the actions of others, termed mirror neurons. Such neurons compose a system of analogue representation that is iconic in origin and underlies the ability to point, imitate and direct the gaze with communicative intention. Thus, these neurons play a role in nonverbal communication (e.g., pointing, showing, demonstrating, expressing emotions) related to pantomime and the ability to imitate the action of others (mimicry). According to McNeill (1992, 1998, 2008), gestures have four dimensions. a) Deictic gestures (pointing) indicate objects or locations in the physical environment. b) Iconic gestures maintain a perceptual relation of similarity between their form and the semantic content of the representation, i.e., the mode of realization of the gesture embodies relatable aspects of the semantic content. c) Metaphorical gestures: here, the semantic content is provided through a metaphor (e.g., joined hands as an expression of “holding” an idea, which would be a metaphor of the idea as an object). d) Rhythmic gestures (beat gestures) are motor movements or rhythmic gestures that do not possess semantic content but are contained in the prosody of semantic content. Following the logic of Corballis (2003) and McNeill, mimic gestures are highly iconic and constitute a basic principle of communicative intention.
Intuitive representations and scientific literacy
Intuitive representations are personal constructions that students bring to their science classes regardless of their age, gender, culture or educational level. Many of these representations have been formed from the experience of individuals with everyday phenomena and guided by perception. They are not easy to identify because they are part of the subject’s implicit knowledge. In addition, they are highly resistant
16 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children and difficult to modify. Therefore, they often become strong barriers to understanding scientific concepts and have remained unaltered by traditional science instruction. These representations are not exclusive to students. Many teachers share the intuitive representations of their students although they may be unaware of it (Carretero et al., 1996, Driver, 1989, Driver and Erickson, 1983, Pozo and Carretero, 1992; Wandersee et al., 1994). There are two main explanations for the resistance to the modification of intuitive representations. One explanation is that representations are closely linked to situations of daily life and thus possess a strong degree of coherence and variable solidity. In such situations, they are appropriate and thus more difficult to modify (Pozo and Carretero, 1992). The other is the lack of prior knowledge. If one does not possess a certain level of knowledge, it is difficult to understand the arguments presented to drive the change of those representations (Strike and Posner, 1985, Chinn and Brewer, 1993, Schumacher et al., 1993).
Intuitive representations regarding sound
Probably because the transmission of sound can be a difficult concept to understand, studies on intuitive representations regarding sound are scarce. Several studies suggest that although children consider sound as an object, they do not award it all the criteria of a physical entity. Mazens and Lautrey (2003) interviewed a group of 89 students 6 to 10 years of age, who were asked to answer a series of questions to determine if they confer to sound physical properties, such as substantiality (i.e., sound has the characteristics of a substance), trajectory (i.e., sound follows a linear trajectory between two points), permanence (i.e., sound travels from its origin in an infinite longitudinal trajectory of time) or weight (i.e., sound is affected by gravity). These authors found that 46% of the children considered that “sound went through holes” in objects, 33% believed that sound travelled directly to the ear or bounced off surfaces before it reached a person’s ear, 20% believed that a sound “continues forever” (i.e., an infinite characteristic of time), and 14% thought that sound had weight. This study showed that when children perceive sound as an object, they attribute the characteristic of substantiality more often than weight or permanence.
Revista de Educación, 381. July-September 2018, pp. 11-34 17 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
A year later, Lautrey and Mazens (2004) analysed the organization of intuitive representations of sound and compared them with such representations of heat. In addition, they investigated the process of conceptual change that occurs in such representations. To this end, they individually interviewed 83 boys and girls (approximately 50% male) 8 years of age to determine if they attributed to sound the properties of objects (such as substantiality, weight, permanence and trajectory) or the properties of physical processes (for example, proximity transmission). These two categories would account for the conceptual change. Their results indicated that children attribute object properties to sound in a hierarchical manner. That is, at the basic level (first hierarchical level), they attribute the quality of permanence. At the next level, they attribute the quality of weight. Finally, at the third level, they attribute the quality of substance. That is, sound is assumed to be a type of matter, which implies that it cannot pass through solids. Lautrey and Mazens’ results also show that these intuitive representations seem to be relatively organized and that conceptual change is a long-term process. For Whittaker (2012), physics is a subject regarding which students have a large number of intuitive representations that are incompatible with scientific knowledge in addition to being deeply rooted. For this author, sound is a good example of such representations because it requires the visualization of a form of energy that is transmitted through an invisible medium in an invisible way. To identify the intuitive representations of students regarding the nature of sound and its transmission, the researcher conducted individual interviews with 28 students 11 to 14 years old. The interview questions included the following: 1. Your teacher is talking in class. How can you hear him or her? 2. The teacher is talking loudly in the classroom next door. How can you hear him or her? 3. If you leave the room and scream, how far will the sound travel? 4. Could you hear someone talking to you if the person were in space? Similarly to the previously described studies (Lautrey and Mazens, 2003, Mazens and Lautrey, 2004), Whittaker concluded that children’s representations of sound are based on a substantialist intuition. That is, the students attribute object properties to sound, such as trajectory and the ability to pass through objects and move. Because few studies have addressed the intuitive representations of students regarding sound, Eshach and Schwartz (2006) used the substance scheme (Reiner et al., 2000) to determine whether such a scheme is present
18 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children in the thinking of high-school students and to examine how students use the characteristics of the scheme to explain sound. They also sought to determine if the characteristics of the substance scheme are used with local coherence (that is, individually) or if more coherent consistencies can be identified at a general level among the properties that students use to explain sound phenomena. To this end, in-depth interviews using open and standardized questions were conducted with ten high-school students. The results show that based on the substance scheme, sound was perceived by the participants as something that can be pushed, rubbed or controlled and as an abnormal substance with respect to its stability, corpuscular nature, additive properties and inertial characteristics. Thus, the students’ representations of sound do not seem to fit the scheme of Reiner et al. (2000) in all aspects. The results also indicate that the students’ representations regarding sound lacked internal coherence. When the students were questioned regarding the explanatory scope of their answers, they were inconsistent. Regarding local and general coherence, the authors report that the students’ concept of sound coincided with the proposal of diSessa (2002), who considers that as a fragmented collection, such representations are loosely connected. Eshach and Schwartz (2006) conclude that sound is perceived only as a type of material (i.e., with qualities of substance). They also note that more research is required to help determine the relevance of the substance scheme. Finally, Perales (1997) studied the understanding of acoustics in students of different educational levels starting with an analysis of scientific content of three types: structural, didactic and sociological. Perales concludes that analogously to light, sound is identified by its cause (emission) and by its effects (detection). However, sound also exists as an entity in space (propagation). Therefore, it represents an advance relative to what occurs with light. In this sense, progression by age or educational level is evident: sound goes from being “a noise” to being a propagation of waves in material space. Perales (1997) considers this outcome to represent an important qualitative advance in student understanding. It reflects evidence of the propagation of sound being attributed to all the particles in the medium. However, the nature of the waves was far from being understood by the students. Finally, it suggests that this progression is manifested in terms of the semantic richness of the term sound, which, while mostly considered as noise by younger students, acquires a variety of adjectives descriptive of its qualities among older students. The significant presence
Revista de Educación, 381. July-September 2018, pp. 11-34 19 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
of “music” as a descriptor of sound among younger individuals was also noteworthy. The described studies provide an overview of the type of representations that students develop regarding the phenomenon of sound. Generally, there is little research on intuitive representations of sound, and there are no studies on preschool populations. Based on the idea that individuals construct intuitive representations regarding the phenomena they observe and that the use of gestures (i.e., body cognition) forms part of these representations, the objective of this study was to identify intuitive representations of children regarding sound and their relation to the formation of scientific knowledge in preschool. To understand the complexity of the research problem and to establish explanatory levels within a given process or situation (Stake, 1998), a systematic, in-depth inquiry is required. Therefore, the analysis is based on a case study. In addition, the observations are detailed and facilitate studying multiple and varied aspects. We examine our subjects and outcomes in relation to others while viewing them within their environments Therefore, our approach is inductive (Pérez Serrano, 1994).
Method
The research is qualitative and corresponds to a case study.
Sample
The sample consisted of six preschool students (two girls and four boys) between 5 and 6 years old. The students belonged to two different regions. Two were from Tesigtan, Puebla, and four were from Mexico City.
Instruments
The data were collected through a semi-structured interview that addressed three aspects of sound: production (i.e., how sound can be produced), perception (i.e., how sound is perceived; who can perceive sound) and propagation (how sound travels; how far it can travel). Table I shows sample questions.
20 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
TABLE I. Structure of the interview
Theme Question Production Look around you, with what things can you make sound? Perception Observe these images (dog, whale, baby, child, chicken, tree, stone, bird, but- terfly, fish, etc.). Who can listen? Why can they listen? How do you know they are listening? If you talk to the butterfly, can it hear you? If you talk to the tree, can it hear you? Propagation The triangle is hit. When you stop listening you ask where the noise went, how far away it went. What happens to that sound, it goes away forever, stops or comes back?
Source: self-made
Because sound involves a level of abstraction that is difficult to explain to young children, graphic and instrumental material was used to contextualize and delimit the interview. This material also supported the process of representational construction of the children by placing them in a specific situation in which they could produce and perceive sound (while the questions were asked). The children could use the aids to make their representations explicit. Table II describes the material used during the session.
TABLE II. Material used during the interview
Theme Material Production Image sheets corresponding to people, parts of the body, instruments or ob- jects with which sound can be produced. Perception Image sheets corresponding to people or objects which can produce sound by themselves (truck, tricycle, dog tree, whale, bird, rain, girl (or boy), baby, stone). Image sheets corresponding to people and animals. Sheet of paper that is placed in front of the face of the interviewer, between him and the child. Marimba with keys of three different materials (rubber, wood and metal). Propagation An alarm clock, four boxes of: wood, glass, cardboard and acrylic (assuming that the clock is placed in each box). Metallic musical triangle. Hose phone. A pair of earmuffs. A drum and a metal bar suspended centimetres from it.
Source: self-made
Revista de Educación, 381. July-September 2018, pp. 11-34 21 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
Process
After we received written authorization from parents and school authorities, the schools provided a space for the interviews. The interview sessions lasted an average of 30 minutes for each child, distributed into two sub-sessions per day, which were audio and video recorded. In principle, the interview sequence was the same for all participants. However, in certain cases, the order was changed with the intention of not losing the attention of the interviewees and understanding their thought processes.
Data analysis
Once the interviews were transcribed, two different analyses were performed in parallel. In the first analysis, categories were constructed to organize the data and interpret them. This categorization was based on grounded theory (Charmaz, 2006), which states that the theory is elaborated and emerges from the analysed data. The explanations and actions of the children were analysed, which thus constituted the response categories of the subjects. Table III describes the criteria used to define the categories. These criteria were based on the ideas that the children expressed during the interviews.
22 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
TABLE III. Categories of interview analysis
Criterion Category Only with letters and words can sound be 1. Word-Sound produced. The sound is produced by the letters; the other sounds are noise. It is possible to produce sound if an action 2. Object-Action-Material is performed on a specific object: i.e., touch, whip, carve, blow, drag, pull and break. The sounds are different because the material that produces it is different. Produce sound with the body and/or per- 3. Body-Sound ceive sound with the ears. The things that have life can listen. However, a tree does not listen. So it is inferred that plants in general are considered objects. 4. Substantialist characteristics The sound is inside the objects, so it is 4.1. Contention necessary to remove it from them and it can be trapped. The sound can pass through the material 4.2. Go through objects that is less hard. The sound follows a definite path to the 4.3. Trajectory objects or ears. Sound can be stopped and continue the trajectory when it occurs again. The “intensity” of the sound depends on 4.4. Force-weight the force with which the object is touched. There are sounds that weigh more than others. That is, the sounds are different depending on the weight.
Source: self-made
In the second analysis, the gestures used by the children to explain their answers were identified and categorized. Alternatively, in certain cases, the gesture was the response itself, following grounded theory. Table IV shows and describes the gestures identified in the interview.
Revista de Educación, 381. July-September 2018, pp. 11-34 23 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
TABLE IV. Dimensions used for gesture analysis
Type of gesture Description Pointing gesture Point with the index finger or with the whole hand, accompanied by the verbal response or not. Iconic gesture Broad movements (usually by hand) that are related to the semantic content of the answers. Metaphorical gesture Related to the semantic content of the response. Beat gesture Motor movements and/or rhythmic gestures that have no relation to the semantic content of the response.
Source: Re-elaborate from McNeil (1992, 1998 and 2008)
Results
Next, the results from the analysis of the interviews are presented. First, we describe the categories identified in the children’s responses, which correspond to the information presented in Table III. Subsequently, the gestures used in the interviews are described. The analysis was performed for the three addressed themes: the perception of sound, the production of sound and the propagation of sound. Table V shows the frequencies observed for each of the analysis categories (Table III) for each of the subjects in the sample.
TABLE V. Frequencies of the categories of analysis present in the representational formation of children with respect to sound according to the theme
Categories C1 C2 C3 C4.1 C4.2 C4.3 C4.4 Theme f f f f f f f Production 1 58 36 1 0 0 0 Perception 0 9 53 0 0 9 1 Propagation 0 10 5 3 2 16 1
Source: self-made Note: C1. Word-sound, C2. Object-action-material, C.3. Body-sound, C4.1. Contention, C4.2. Go through objects, C4.3. Trajectory, C4.4. Force-weight.
24 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
Figure I shows which categories were used for each topic and their percentages (calculated from the total of the children’s responses).
FIGURE I. Percentage of categories of analysis by theme
Of a total of 205 categorized answers (Figure I), it can be observed that category C1 only occurs in one case, probably because the student was in the process of learning to read and write and considered that sound could only be made with words. Categories C2 (37.56%) and C3 (45.08%) had the highest percentage of sound representation use. They were employed to explain the production and perception of sound in all cases. Category C4 (16.1%), which includes Subcategories C4.1, C4.2, C4.3 and C4.4, occurs in the representations of all the students. Although it is used to a lesser extent, it is more often linked with the themes of the perception and propagation of sound. It is necessary to note that Case 2 did not yield enough information. Thus, in this case, only Categories C2 and C4.3 were used to explain sound production and propagation. The categories account for the answers the children provided to all the interview questions. However, as mentioned, many of the children used gestures to answer the questions.
Revista de Educación, 381. July-September 2018, pp. 11-34 25 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
The second analysis identified the gestures that the children used to answer the interview questions, which on several occasions constituted the answer itself. Table IV describes the frequencies of the gestures used in each of the categories of analysis and for each theme. This analysis shows that the children only used two gesture types: deictic and iconic.
TABLE VI. Frequency of the gestures used in the categories of analysis present in the represen- tational formation of children with respect to sound according to the theme
Categories and type of gesture Theme C1 C2 C3 C4.1 C4.2 C4.3 C4.4 PG IG PG IG PG IG PG IG PG IG PG IG PG IG Production 2 1 27 34 12 15 7 1 3 1 7 8 Perception 2 3 38 1 2 2 1 2 1 Propagation 1 1 2 1 2 5
Source: self-made Note: PG pointing gesture; IG iconic gesture.
Figure II shows the percentages (calculated from the total gestures recorded from the sample) and gesture types that were used by category and topic. Figure II shows that of the 182 analysed gestures, Categories C2 (37.57%) and C3 (36.46%) exhibited the highest use frequency. Category C4 generally occupied third place (24.31%) and is distributed mainly between subcategories C4.1, C4.2 and C4.3. The type of gesture most used by the children was deictic (pointing to things; 54.7%), followed by iconic, i.e., broad movements related to the content (39.23%). Gestures were used more frequently (65.19%) in connection with the theme of sound production, followed by the theme of sound perception (28.18%) and finally the theme of sound propagation (6.63%). Importantly, the other forms of gesture identified in the literature, i.e., metaphorical and rhythmic, did not appear in the children’s responses in any of the analysed categories probably because metaphorical gestures involve a more abstract construction than deictic and iconic gestures can express. That is, instead of representing specific objects or events, metaphorical gestures represent concepts and relationships. Therefore, when considering sound with characteristics of an object, children are
26 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children incapable of using a metaphorical element to describe such a sound. Unlike an iconic gesture, a rhythmic gesture is not communicative. Such representations are not linked to semantic content but to oral production.
FIGURE II. Percentage of use of gestures by subject and category
The data analysis revealed that, in all cases, the children had a representation based on actions that produce sound (C2) and the way in which they perceived sound (C3). However, a substantialist representation of sound was in the process of being constructed (C4), that is, that sound has properties analogous to those of objects (e.g., weight or a defined trajectory). Subcategories C4.2 and C4.3 are present for all subjects, Subcategory C4.1 only appears for one student, and finally, Subcategory C4.4 appears for two students. This idea of the substantialization of sound is similar to that reported in other investigations, although the characteristics of the analysed samples, in particular age, are different. Regarding the substantialist characteristic, other studies suggest that although children consider sound an object, all the criteria of a physical entity do not apply (Mazens
Revista de Educación, 381. July-September 2018, pp. 11-34 27 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
and Lautrey, 2003). A similar phenomenon can be observed in the results of this study regarding the type of substantive response that the students provided. In the analysed answers, this representation is supported by elements centred on the characteristics of materials and the actions that the students perform to produce or perceive sound. In the case of gestures, it was generally observed that the gesture most used in all cases to express representations was the deictic gesture, primarily regarding sound production and sound perception, with the use frequency decreasing for sound propagation. This is because at a hierarchical level, the deictic gesture is the most basic dimension (McNeill, 1992, 1998, 2008) and can replace verbal communication (e.g., pointing, showing, expressing emotions). Category C1 was expressed in Case 1. Category C2 was expressed in Cases 1, 3, 4, 5 and 6. Regarding Category C4, Subcategory C4.1 was expressed in Cases 1, 3, 4, 5 and 6, Subcategory C4.2 appeared in Cases 1, 3, 4 and 6, Subcategory C4.3 was expressed in all cases, and Subcategory C4.3 was only expressed in Case 6. In the case of iconic gestures, the frequency of use was greater in explaining sound production, corresponding to C1 for Case 1, C2 for all cases and C3 in Cases 1, 3, 4, 5 and 6. In addition, although in all cases Category C4 is expressed, Subcategory C4.1 appeared in Cases 1, 3, 4, 5 and 6, Subcategory C4.2 appeared in Cases 1, 3, 4 and 6, Subcategory C4.3 was present in all cases, and Subcategory C4.4 only appeared in Case 6.
Discussion and Conclusion
In the course of the interviews, it was possible to observe in the children the existence of a representation of sound that had two main characteristics. The first was to attribute to sound the properties of an object, which corresponds to analysis categories C2 and C3. This characteristic appeared in all cases, whereby it was believed that sound cannot pass through a solid. Therefore, a sound can be “contained” inside a solid, for example, inside a container, and for a sound to spread, a specific action and a space without obstacles are required. The second characteristic was that to explain observations that initially contradicted the initial representation, the children seem to construct a synthetic mental representation (Vosniadou, 1994). This characteristic was observed
28 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children in Cases 1, 3, 4 and 6. In these cases, the students represented sound as a substance but one that could pass through a solid. Thus, here, sound was awarded different properties unique to solids because the children believed that sound could pass through less hard or less heavy objects. However, the children did not specify how this phenomenon occurs, which indicates that this concept was in the process of construction. Both characteristics respond to phenomenological experiences guided essentially by the students’ perception. The representations of these children are based on a materialist intuition that suggests a substantialist structure as the basis of their representations regarding sound. This outcome suggests that the formation of intuitive representations is based on the experience with everyday phenomena, guided by perception. Such intuitions are not easy to identify because they are part of the implicit knowledge of the subject (Carretero et al., 1996, Pozo and Carretero, 1992). In two cases, the idea that sound cannot be seen was observed. However, throughout the interviews, it was confirmed that although the children seemed to distance themselves from the idea of sound as an object, they maintained the substantialist representation because they continued to grant sound properties that explain its movement although it is invisible to them. In two other cases, the children mentioned or perceived vibration during the production of sound. However, they did not manage to articulate this concept in their representations. That is, they intuited that there is a relationship between vibrations and sound. However, they were not yet able to make that relationship explicit in an organized way and integrate it into their representations. Finally, it is important to note that the results were consistent between the analyses of the verbal responses and the gestural responses. The main categories used to explain sound production and perception were related to objects and actions as well as to the body (C2 and C3). Although it also appears in connection with these themes, the substantialist conception (C4) was linked somewhat more strongly with sound propagation. One limitation of this study was the number of participants, which restricts the generalization of the results. However, the study of representations in students continues to be a relevant topic. A particular contribution of this preliminary study is that it addresses a topic little discussed in connection with children of this age and considers the gestures such subjects use. Thus, we were able to analyse the intuitive
Revista de Educación, 381. July-September 2018, pp. 11-34 29 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
representations that children constructed to explain phenomena observed in their daily life and the understanding they achieve when, for example, specific teaching activities are used. As previously mentioned, the representations that the students construct strongly influence subsequent learning. In addition, many of these representations have a strong resistance to modification because they are closely linked to situations of everyday life and thus possess a strong degree of coherence and solidity. Therefore, additional research is required to improve our understanding of how representations are constructed and how they evolve during schooling. That is, the sample should be expanded not only in quantity but also to include different ages to understand how sound goes from being a noise to a propagation of waves in material space and to understand the basis on which this transformation occurs. In addition, researchers should compare their results with those for other countries with the purpose of determining the scope of a pedagogical proposal for teaching science at the elementary level of education.
References
Alibali, Martha W. and Mitchell J. Nathan (2012), “Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures”, Journal of the Learning Sciences, vol. 21, núm. 2, pp. 247-286. Barman, Charles R, Natalie S. Barman y Julie A. Miller (1996), “Two teaching methods and students’ understanding of sound, “School Science and Mathematics”, vol. 96, n. pp. 63-67. Calvo, Paco and Tony Gomila (Eds.). (2008), Handbook of cognitive science: An embodied approach, USA, Elsevier. Carretero, Mario, Maite Baillo and Margarita Limón (1996), Construir y enseñar: las ciencias experimentales, Argentina, Aique. Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. London: Sage. Chi, T. Michelene and Rod D. Roscoe (2002), “The processes and challenges of conceptual change”, in Limón and Mason (Eds), Reconsidering conceptual change: Issues in theory and practice, pp. 3-27. New York, NY: Springer.
30 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
Chinn, Clark and William Brewer (1993), “The role of anomalous data in knowledge acquisition: A theoretical framework and implications for science instruction”, Review of educational research, vol. 63, n. 1, pp. 1-49. Clark, Andy and David Chalmers (1998), “The extended mind”, Analysis, vol. 58, n. 1, pp. 7-19. Clement, John, David E. Brown and Aletta Zietsman (1989), “Not all preconceptions are misconceptions: finding ‘anchoring conceptions’ for grounding instruction on students’ intuitions.” International journal of science education, vol. 11, n. 5, pp. 554-565. Cook, Michelle Patrick (2006), “Visual representations in science education: The influence of prior knowledge and cognitive load theory on instructional design principles”, Science education, vol. 90, n. 6, pp. 1073-1091. Corballis, Michael C. (2003), “From mouth to hand: gesture, speech, and the evolution of right-handedness”, Behavioral and Brain Sciences, vol. 26.02, pp. 199-208. Delval, Juan (2007), “Aspectos de la construcción del conocimiento sobre la sociedad (Aspects of the construction of knowledge about society)”, Educar, vol. 30, pp. 45-64. diSessa, Andrea (2002), “Why conceptual ecology is a good idea”, en Limon, y Mason (Eds.), Reconsidering conceptual change: Issues in theory and practice, New York, NY: Springer, pp. 28–60. Driver, Rosalind (1989), “Students’ conceptions and the learning of science”, International Journal of Science Education, vol. 11, n. 5, pp.481-490. Driver, Rosalind and Gaalen Erickson (1983), “Theories-in-action: Some theoretical and empirical issues in the study of students’ conceptual frameworks in science”, vol. 10, pp. 37-60. Duschl, Richard A. and Richard E. Grandy (2008), Teaching scientific inquiry: Recommendations for research and implementation. USA, Sense Publishers. Edwards, Laurie D. (2009), “Gestures and conceptual integration in mathematical talk”, Educational Studies in Mathematics, vol. 70, n. 2, pp. 127-141. Eshach, Haim and Judah L. Schwartz (2006), “Sound Stuff? Naïve materialism in middle-school students’ conceptions of sound”, International Journal of Science Education, vol. 28, n. 7, pp. 733-764.
Revista de Educación, 381. July-September 2018, pp. 11-34 31 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
Gibbs Jr, Rimond, W. (2005), Embodiment and cognitive science, Cambridge, Cambridge University Press. Hluštík, Petr, Solodkin, Ana, Noll, Douglas C. and Small, Steven (2004), “Cortical plasticity during three-week motor skill learning” Journal of Clinical Neurophysiology, vol. 21, pp. 1–12. Hostetter, Autumn B. and Martha W. Alibali (2008), “Visible embodiment: Gestures as simulated action”, Psychonomic Bulletin Review, vol. 15, n. 3, pp. 495-514. Lautrey Jacques and Mazens, Karine (2004), “Is children’s naive knowledge consistent? A comparison of the concepts of sound and heat”, Learning and Instruction, vol. 14, n. 4, pp. 399-423. Mazens, Karine and Lautrey Jacques (2003), “Conceptual change in physics: children’s naïve representations of sound”, Cognitive Development, vol. 18, pp. 159–176. McNeill, David (1992), Hand and mind: What gestures reveal about thought, USA, University of Chicago press. McNeill, David (1998), “Speech and gesture integration”, New Directions for Child and Adolescent Development”, vol. 79, pp. 11-27. McNeill, David (2008), Gesture and thought, USA, University of Chicago Press. Moreira, Marco Antonio, and Ileana María Greca (2005), “Cambio conceptual: análisis crítico y propuestas a la luz de la teoría del aprendizaje significativo”, Ciência & Educação. Bauru, vol. 9, n. 2, pp. 301-315. Nersessian, Nancy (1999). Model-based reasoning in conceptual change, en Magnani, L. Nersessian, Nancy, Thagard, Paul (eds.) Model-based reasoning in scientific discovery, USA, Springer, pp. 5-22. Osborne, Roger and Peter Freyberg (1998), El Aprendizaje de las ciencias: implicaciones de las” ideas previas” de los alumnos, vol. 121, Madrid, España, Narcea Ediciones Perales Palacios, Francisco Javier (1997), Escuchando el sonido: concepciones sobre acústica en alumnos de distintos niveles educativos, Enseñanza de las Ciencias, vol. 15, n. 2, pp. 233-247. Pérez Serrano, Gloria (1994), Investigación cualitativa: retos e interrogantes. Técnicas y análisis de datos, México, La Muralla. Piaget Jean e Barbel Inhelder ([1966] 2008), The psychology of the child, New York, Basic books.
32 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
Postigo, Yolanda and Asunción López-Manjón (2012), “Students’ Conceptions of Biological Images as Representational Devices”, Revista Colombiana de Psicología, vol. 21, n. 2, pp. 265-284. Pozo Juan Ignacio y Mario Carretero (1992), “Causal theories, reasoning strategies, and conflict resolution by experts and novices in Newtonian mechanics” en Andreas Demetriou, Anastasia Efklides, Michael Shayer (eds) Neo-Piagetian Theories of Cognitive Development. Implication and Applications for Education, Londres: Routledge, pp. 231-55. Pozo, Juan Ignacio (2003), Adquisición de conocimiento: cuando la carne se hace verbo, Madrid, Morata. Pozo, Juan Ignacio and Miguel Ángel Gómez-Crespo (1998), “Aprender y enseñar ciencia: del conocimiento cotidiano al conocimiento científico”,Madrid, Morata. Reiner, Miriam, James Slotta, Michelene T. H. Chi and Lauren B. Resnick (2000), “Naive physics reasoning: A commitment to substance-based conceptions”, Cognition and instruction, vol. 18, n. 1, pp. 1-34. Rowe, Meredith L, Şeyda Özçalışkan and Susan Goldin-Meadow (2008), “Learning words by hand: Gesture’s role in predicting vocabulary development”, First language, vol. 28, n. 2, pp. 182-199. Rowe, Meredith L. and Susan Goldin-Meadow (2009), “Differences in early gesture explain SES disparities in child vocabulary size at school entry”, Science, vol. 323, pp. 951–953. Schumacher, G. M., Tice, S., Wen Loi, P., Stein, S., Joyner, C., y Jolton, J. (1993), “Difficult to change knowledge. Explanations and interventions. En Proceedings of the Third International Seminar on Misconceptions and Educational Strategies in Science and Mathematics”, Ithaca: Cornell University, en: https://www.google.com.mx/url?sa=t&rct =j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUK Ewi5pbvxiZPSAhXJsFQKHTNWAOAQFggZMAA&url=http%3A% 2F%2Fwww.mlrg.org%2Fproc3pdfs%2FSchumacher_Knowledge. pdf&usg=AFQjCNFXl-ZtakrFEGrGN6-f2s1lZR55Bw&sig2=V3BD2n14s C2Rnp5zEmzw-w, (date of reference: 4 February, 2017). Stake, Robert E (1998), Investigación con estudio de casos. Madrid, Ediciones Morata. Strike, Kenneth A. and George J. Posner (1985), A conceptual change view of learning and understanding, USA, Academic Press. Thompson, Evan, Adrian Palacios and Francisco Javier Varela (1992), “On the ways to color”, Behavioral and Brain Sciences, vol. 15, n. 1, pp. 56-74.
Revista de Educación, 381. July-September 2018, pp. 11-34 33 Received: 02-05-2017 Accepted: 16-03-2018 Velázquez-Olmedo, C. E., Gallegos-Cázares, L., Calderón-Canales, E. Construction of intuitive representations about sound in preschool children
Tytler, Russell and Vaughan Prain (2010), “A framework for re-thinking learning in science from recent cognitive science perspectives” International Journal of Science Education, vol. 32, núm. 15, pp. 2055-2078. Wandersee, James, Joel Mintzes and Joseph Novak (1994), “Research in alternative conceptions in science”, in D. Gabel (ed.), Research Handbook on Research on Science, Teaching and Learning New York, N.Y.: McMillan Pub, pp. 177-210. Whittaker, Andrew (2012), “Pupils Think Sound Has Substance--Well, Sort of”, School Science Review, vol. 346, pp. 109-111. Wilson, Margaret (2002), “Six views of embodied cognition”, Psychonomic bulletin review, vol. 9, n. 4, pp. 625-636.
Contact address: Mtra. Claudia-Elena Velázquez-Olmedo. Universidad Nacional Autónoma de México. Instituto de Ciencias Aplicadas y Tecnología. Grupo de Cognición y Didáctica de las Ciencias. ICAT, Avenida Universidad Nº3000, Ciudad Universitaria. Delegación Coyoacán, Ciudad de México, CP. 04510. E-mail: [email protected]
34 Revista de Educación, 381. July-September 2018, pp. 11-34 Received: 02-05-2017 Accepted: 16-03-2018 Linking the University of Granada CertAcles listening test to the CEFR1
Vinculación de la prueba de comprensión oral del examen CertAcles de la Universidad de Granada con el MCER
DOI: 10.4438/1988-592X-RE-2017-381-380
Caroline Shackleton Universidad de Granada
Abstract As part of the ongoing collection of validity evidence to support the interpretation and use of the UGR bi-level English test scores this study aims to formally map the oral comprehension part of the test to the Common European Framework of Reference (CEFR). The advice provided by the Council of Europe (2009) was followed in order to determine cut scores which are representative of CEFR B1 and CEFR B2 levels. The test used in the study was from the March 2017 administration (N=464) which was shown to be a reliable and appropriate measurement instrument. The study involved ten participant judges who, after a CEFR familiarisation session, took part in standard setting sessions. Two methods of standard setting were employed. The Basket method was used mainly in order to familiarise the participants with the test items and was analysed using a Many-Facet Rasch measurement model. The Bookmark method allowed for the incorporation of item difficulty parameters produced by a Rasch analysis of test scores. The final median cut scores of the judges using a response probability of .67 were mapped back to the Rasch theta ability scale in order to determine the raw scores which represent the two CEFR ability levels. Several other validity and reliability checks were also carried out following CoE (2009) and transparent documentation on the whole process is provided. By using Rasch common item equating, the scores are reproducible on future versions of the test and so support the generalisation inference of the UGR test’s validity argument.
(1) Acknowledgements: The author would like to thank the Centro de Lenguas Modernas of the University of Granada for supporting this study, and the two anonymous reviewers for their helpful comments and suggestions.
Revista de Educación, 381. July-September 2018, pp. 35-60 35 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
Keywords: proficiency tests (language), standard setting, validity, item response theory
Resumen Este estudio, como parte del proceso de validación fehaciente del uso e interpretación de las puntuaciones del examen de inglés binivel de la Universidad de Granada (UGR), pretende relacionar, oficialmente, la prueba de comprensión oral de este examen con el Marco Común Europeo de Referencia (MCER). Con tal fin seguimos las recomendaciones del Consejo de Europa (2009) para determinar unos puntos de corte representativos de los niveles B1 y B2 del MCER. La prueba utilizada en el estudio fue la convocatoria de marzo del 2017 (N=464), que demostró ser un instrumento de medición fiable y apropiado. En el estudio participaron diez expertos, quienes, tras una sesión para familiarizarse con el MCER, realizaron varias sesiones de fijación de puntos de corte. Se utilizaron dos métodos: el método Basket, o de la cesta, para familiarizar a los participantes con los ítems de la prueba. después del cual se analizaron los resultados empleando un modelo de múltiples facetas de Rasch (MFR); por otro lado, el método Bookmark, o del marcador, que incorporó parámetros de dificultad del ítem, resultantes de un análisis Rasch de los puntajes de la prueba. La mediana final de los puntos de corte aportados por los expertos se calculó empleando una probabilidad de respuesta de 0.67, y se transfirió a la escala de capacidad zeta del modelo Rasch para determinar la puntuación bruta que representarán los dos niveles de dominio del MCER en cuestión. También se realizaron otros controles de fiabilidad y de validez para proporcionar una documentación trasparente sobre el proceso completo, recomendado por el Consejo de Europa (2009). El método de equiparación de puntuaciones del modelo Rasch permite que los puntajes sean reproducibles en futuras versiones de la prueba y, por tanto, apoyan la inferencia de generalización del argumento de validez para la prueba binivel de la UGR.
Palabras Clave: pruebas de competencia lingüística, fijación de puntos de corte, validez, la teoría de respuesta al ítem
Introduction
As part of continuing policy adaptation to the Bologna process and the introduction of the European Higher Education Area, recent Spanish Ministry of Education policy has placed emphasis on communication in a foreign language as a fundamental competence for enabling
36 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR greater student mobility and employability. To this end, any national test must therefore be designed in such a way as to allow the linking of candidate performance to CEFR competence levels if it is to respect principles of transparency, comparability and coherence. Such principles are unquestionably of the utmost importance in the promotion of educational and professional mobility within the EU in accordance with current European plurilingualism policies. Indeed, the recommendation on the use of the CEFR by the Council of Ministers includes the call for countries to:
‘…ensure that all tests, examinations and assessment procedures leading to officially recognised language qualifications take full account of the relevant aspects of language use and language competences as set out in the CEFR, that they are conducted in accordance with internationally recognised principles of good practice and quality management, and that the procedures to relate these tests and examinations to the common reference levels (A1-C2) of the CEFR are carried out in a reliable and transparent manner.’ (Council of Europe, 2008, p.4)
In Spain, many university language centres have responded to this call by providing accreditation exams following international guidelines in order to produce valid and reliable CEFR-related tests. Furthermore, in an attempt to better coordinate and regulate such efforts, the Association of Higher Education Language Centres (ACLES) has provided a procedure for nationally recognised, CEFR-related certification: the CertAcles exams. These exams have not only been approved by the Committee of University Rectors (CRUE 08/09/2011) but have also been recognised at an international level by the CercleS organisation since 2014. The CertAcles model (http://www.acles.es/es) requires that each language use ability is tested separately, should be developed from CEFR informed specifications, and that quality control information following the recommendations of the standards (AERA, APA and NCME, 2014) be provided. The University of Granada’s Centro de Lenguas Modernas (CLM) administers a bi-level B1/B2 CertAcles exam three times a year, provided free of charge for University of Granada (UGR) students. Developed in conjunction with EALTA expert member Dr. Rita Green, and initiated in 2009, it is now a stable exam system which follows all the validity checks
Revista de Educación, 381. July-September 2018, pp. 35-60 37 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
recommended by ACLES. The present study will outline and report on the procedures for CEFR linking and cut score determination currently in use on the listening section of the UGR bi-level test at B1 and B2 levels.
Literature review
The CEFR and its six descriptor scales (A1 through to C2) has become the standard currency in language proficiency levels both in Europe and beyond (Deygers, Zeidler, Vilcu, & Carlsen, 2017; Figueras, 2012). Through the association of a test score with a CEFR level, meaning may be added for test users (Kane, 2012). For example, if a score reports B1 proficiency in listening, the score user is informed about the kind of activities a person receiving such a result should be able to perform (Tannenbaum & Cho, 2014). As such, tests which claim to measure CEFR competencies must provide evidence to support these claims: the test must be valid. Validity is seen not as an innate property of the test — a test cannot be validated (Chapelle, 2012; Cizek, 2016) — but rather as a unitary, multi- faceted concept which draws upon multiple sources of evidence in order to substantiate the inferences made about a candidate’s performance and so justify the resulting decisions taken on the basis of test scores (Messick, 1989). Evidence to support the interpretation and use of test scores is necessary, and a validity argument-based approach, which provides a framework to guide testing bodies on the types of evidence to be collected, is therefore recommended (AERA, APA and NCME, 2014; Bachman, 2005; Bachman & Palmer, 2010; Chapelle, 2012; Kane, 2012). The types of evidence to be accumulated should be driven by test purpose (Fulcher & Owen, 2016), and in the European context such a framework has been provided by ALTE (2011) as an aid to testing bodies in the design and development of CEFR-related tests. This document outlines evidence which should be collected throughout the test development cycle (see figure 1) with the goal of building test validation into the whole process of test design and implementation. In contrast to recommendations by Bachman (2005) and Bachman and Palmer (2010), the chain of reasoning begins with the test construct rather than the decisions based on test scores. As such, it links performance on test tasks to an inference about candidate’s language ability beyond the test
38 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR and the argument ends with the ‘extrapolation inference’. However, the world beyond the test remains unspecified: it is a general claim linking to the CEFR. We can therefore argue that the ALTE model directs our interest to the interpretation of the score, which is to ask, ‘can the score be interpreted as a CEFR proficiency level?’. This would follow the conceptualisation of validity proposed by Cizek (2016), who provides a framework that differentiates the validation of test score inferences and justification for test use.
FIGURE I. Chain of reasoning in a validity argument
Observation Evaluation Generalisation Extrapolation What to observe? How can we score Are scores consistent What is the candidate s ability How? what we observe? and interpretable? in the world beyond the test?
Real world Test construction Test performance Measure Test score (target situation of use) use + Source: ALTE (2011, p. 15)
Within such an argument approach, ‘validity evidence of the cut score is an essential component’ (Papageorgiou & Tannenbaum, 2016, p.110). Candidates must meet the cut score to demonstrate that they have reached the required proficiency level, and the results of a test would clearly be open to question unless such a cut score was appropriately set. Here, the Council of Europe (2009) provides a manual which outlines the process that testing bodies should follow. The linking process consists of the following four interrelated activities necessary for relating test scores to the CEFR:
■ Familiarisation. Members of any linking panel must be familiar with the content of the CEFR and its scales. ■ Specification. This should include a detailed description of the test and its relationship to CEFR categories in order to build a linking claim about the content relevance of the exam to the CEFR. ■ Standardisation and Standard setting. For any test, a cut score or score which is needed for passing the test needs to be decided.
Revista de Educación, 381. July-September 2018, pp. 35-60 39 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
■ Empirical validation. CoE (2009) outlines three types of validity evidence – procedural, internal, and external – which must be provided as part of the standard setting process.
Since its publication numerous testing bodies have followed the advice of the manual in order to claim CEFR-linkage. Most international English language proficiency exam providers have carried out such studies (see for example Brunfaut & Harding, 2014; Kanistra & Harsch, 2017; Tannenbaum & Wylie, 2008) and smaller national exam providers also claim CEFR calibration by following the steps in the manual (see for example Downey & Kolias, 2010; Kantarcioglu, Thomas, O’Dwyer, & O’Sullivan, 2010). The main focus of these studies is on the standard setting procedure and its empirical validation. As CoE (2009, p.11) put it ‘the crucial point in the process of linking an examination to the CEFR is the establishment of a decision rule to allocate students to one of the CEFR levels on the basis of their performance in the examination’, that is to say the determination of the cut score. In sum, a valid claim for CEFR linkage must show that the test is reliable and representative of proficiency at a CEFR performance level as ‘if an exam is not valid or reliable, it is meaningless to link it to the CEFR’ (Alderson, 2012). Consequently, the process is normally carried out late in the test development cycle. The UGR test provides just such evidence, and the present study will focus on the official linking process as laid out by CoE (2009) in order to create valid cut score decisions for B1 and B2 mastery for its bi-level listening paper.
Methodology
The literature classifies standard setting techniques into two main types: test- and examinee-centred. For the present study, no information about candidates taking the test was available, and it was therefore decided that the main study would use a test-centred method. Nevertheless, information about a small number of test takers previously enrolled on an exam preparation course at the CLM was also used for comparison in order to give some supporting triangulation evidence (and therefore evidence towards external validity). Of the various test-centred methods, it was decided not to use Angoff- type probability methods. Not only are these methods time-consuming,
40 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR but previous studies report problems concerning judges’ inability to understand and correctly articulate conditional probability (Ferrara & Lewis, 2012; Hambleton & Jirka, 2006; Reckase, 2010). It was decided instead to use a combination of two other methods currently popular in the literature: the Basket method and the Bookmark method. The basket method is effectively an item descriptor matching method specifying the level of competence necessary to answer each item on the test. Judges analyse test items and answer the question, ‘what CEFR level must a test taker be to answer this item?’ It is considered to be the most simple and practical of all standard setting methods and is one which both reflects the importance of the performance level descriptors and places emphasis on test content. However, no information is provided about the difficulty of the items and as such, one of the main problems is a lack of consistency when comparing judgements to empirical difficulty measures (Kaftandjieva, 2010). However, the method was used in the present study as a means of familiarizing judges with the test content and for promoting discussion, rather than to set the final standard. The results of this part of the study were analysed using a Many-Facet Rasch measurement model (MFRM) in the programme FACETS (Linacre, 2017). While it has been argued that MFRM is not appropriate at the later stages of standard setting, where efforts are being made to reach consensus (Eckes, 2009), this was deemed irrelevant to the present study as it would only be used for analysis at the initial familiarisation stage. If the process of setting cut scores is to have real meaning, judges must be given statistical information about test items. In the Bookmark method (Mitzel, Lewis, Patz, & Green, 2001), items are placed in an ordered item booklet (OIB), ordered by their difficulty parameters. Judges move through the OIB booklet from easiest item to most difficult, placing a bookmark at the point where they believe a minimally competent candidate will have less than a specified probability of giving a correct response. The candidate is considered to have the ability to master an item in probabilistic terms, known as the response probability (RP), which should be decided in advance, but it is often set at 0.67, a 67%, or 2/3, chance of the candidate answering the item correctly. The bookmark is placed between two items on the latent scale represented by the Rasch logit difficulty parameters of the items β( -parameters). Finally, the median of judges’ scores is taken to give the group standard on the latent variable. The Bookmark method has the advantage of both being easily
Revista de Educación, 381. July-September 2018, pp. 35-60 41 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
used with tests developed using Rasch measurement, where test taker ability and item difficulty are placed on the same scale, and of clearly reflecting the continuous nature of the CEFR scales, making it ideal for use in the present study.
Instruments
The test used in the present study was taken from the March 2017 administration. The listening paper (42 items) consists of five tasks - two at B1, one at B1/B2 and two at B2 level. All tasks were developed following the test specifications and in collaboration with our expert consultant and may therefore be considered as having been expertly judged to have the correct level of difficulty. Furthermore, they have all been previously piloted (N = 144 to 434) and items found not to function correctly have been removed or revised. The construct is well sampled based on B1 and B2 CEFR descriptors and a mix of test methods are used.
FIGURE II. Item variable map
PERSON - MAP - ITEM
.#### T .### L39 2 .### + L42 ##### S #### L38 .########### S #### L29D 1 ##### + L9A .####### L22 L37 ### L25 L26C L33A ######## L16 L28A ### M L12B L18 L19 L32D 0 .######## +M L10B L11D L24 L34 L41 ############ L1C L30B L4A .#### L27B L36 L6D ############ L21 L23 L7B .##### L20 -1 .########### + L31C #### .#### SSL3H L8D .##### L15B L2F ##### L13C -2 ### + L14C .# L17 L5I #T .# T -3 + .
. -4 #+
Source: Created by the author
42 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
A classical test analysis of the results (where N = 464) gave a Cronbach’s Alpha of 0.934 (SEM = 2.5), evidencing test reliability. The Rasch analysis of test scores showed the test to be unidimensional, with Rasch Infit Mean Square (MNSQ) statistics for all the items falling between the acceptable values of 0.74 and 1.24. The item separation is 10.2 and item separation reliability is 0.99. Therefore, the test can be said to include a good spread of item difficulties and we can be confident that these difficulty estimates are reproducible. The item variable map reproduced in figure II shows ordered item difficulty along with candidate ability. This information was used to create the OIB for the Bookmark method study. Here, the mean logit values for item difficulty and candidate ability are very similar, demonstrating that the test is appropriate in terms of difficulty for the population tested. As the present study does not intend to investigate the test itself, the above information is considered sufficient as evidence that the measurement instrument used in the study is both reliable and appropriate. In order to provide evidence for procedural validity a post standard setting questionnaire was administered following Cizek (2012). The questionnaire contained a four-point likert scale to eliminate the possibility of neutral responses. This allowed for the collection of judge’s opinions concerning their understanding of the whole process and their overall beliefs about the final cut score judgements.
Subjects
It is important that the participants have the relevant expertise in both the CEFR framework and hence its performance level descriptors, as well as in the instruction and assessment of the language being tested (Tannenbaum & Cho, 2014). This is indeed the case of the ten judges who took part in the study. As table I shows, all are experienced TEFL teachers at the CLM and are well-acquainted with the CEFR. Furthermore, they are familiar with the examinee population, and appreciate the consequences of the recommended cut scores.
Revista de Educación, 381. July-September 2018, pp. 35-60 43 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
TABLE I. Background information about judges
Number of Number of hours Judge Sex Qualifications years tea- specific training ching TEFL CEFR/Assessment 1 Female Degree, MA 35 300+ 2 Male Degree 28 80 3 Male Degree, 35 100+ Postgraduate TEFL diploma 4 Male Degree, MA, CELTA 16 70 5 Male Degree 22 70 6 Female Degree, MA 36 400+ 7 Female Degree, PGCE, PhD 28 750+ 8 Male Degree, MA, CELTA 18 500+ 9 Female Degree, PGCE, MA 28 750+ 10 Female Degree 22 500+
Source: Created by the author
Results
The results of each phase of the standard setting sessions, including those results pertinent to the familiarisation stage and external validation, will now be reported.
Familiarisation
In preparation for the study, all the judges were asked to carry out the training sessions for listening provided by the Ceftrain project (see http:// www.helsinki.fi/project/ceftrain/index.html). A CEFR familiarisation descriptor sorting exercise was carried out in which the judges were given a mix of listening descriptors from the CEFR scales and asked to allocate a CEFR level for each one. The results gave a Cronbach’s Alpha of 0.98. Table II shows the correlation between CEFR descriptors and participants’ judgment using a non-parametric Spearman’s rank correlation coefficient. These results were presented to the participants
44 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR and further discussion took place in order to reach consensus on salient features of CEFR levels.
TABLE II. Correlation between CEFR descriptors and judges allocations
Judge 1 2 3 4 5 6 7 8 9 10 .851 .875 .835 .889 .896 .836 .957 .985 1.00 .870
(Note: all correlations were significant at p ≤ .01) Source: Created by the author
Basket method of standard setting.
Judges were first given an input session on the exam construct and test specifications, which was followed by an explanation of the Basket method itself. Figure III shows an MFRM analysis of the results from all judges for each item on the test. The analysis allows for multiple aspects of the judgements to be taken into account, and calibrates items, judges and the rating scale onto the same equal-interval scale. The scale used to represent the minimum CEFR level that a candidate should have in order to answer each item is as follows: 1=A2, 2=minimum B1, 3=strong B1, 4= minimum B2, 5=strong B2, 6=C1. The results from the two rounds differed only minimally, showing that the judges were confident in their decisions. It can be clearly seen that the judges believed the content of the items to be well spread along the ability scale from CEFR B1 to CEFR B2. The Rasch Andrich thresholds showed that judges placed items into distinct categories, which increase in difficulty level. These categories are very much in agreement with the test developers’ intentions. The judges differed slightly, with judge 10 placing the items at a marginally lower difficulty level. However, the MFRM analysis takes such variations into account and, by making adjustments for the leniency or severity of the judges, is able to give a ‘fair average’ score for each item. Using the Rasch Thurston thresholds, which measure the boundary at which an item has a 50% chance of being rated in a higher or lower category (i.e. the 50% (median) cumulative probability threshold), we reach preliminary cut scores of 13 for B1 and 34 for B2.
Revista de Educación, 381. July-September 2018, pp. 35-60 45 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
FIGURE III. Vertical ruler from FACETS analysis for Basket method results
+------+ Measr +rater +item -round Scale +-----+------+------+------+----- +8++++(6) *
+7++++ --- * +6++*++
** +5++++5
* +4++*++ * ---
+3++*** ++ ** * +2++++4 * ** +1++++ 412 1 2378 --- 0+ +* ++ 569
+-1+ +* ++ 10 ****** 3 * +-2+ +* ++
** --- +-3+ + ******* ++ * * +-4+ + *** ++
+-5+ +++ *2
+-6+ +++ (1) +-----+------+------+------+----- Measr +rater *=1 -round Scale +------+
Source: Created by the author
Bookmark method of standard setting.
Before moving on to the main study, participants were given feedback, including impact data, from the Basket method. Discussion was finalised with the general conclusion that the judges felt the B2 cut score to be rather high (only 81 students, or 17%, would receive a B2). At this point, it was explained that these would not be the final cut scores and that the Basket exercise was simply intended to allow the participants to become familiar with the test before seeing the real item difficulty values that would be used to allocate cut scores.
46 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
The bookmark method was then explained to the participants and they were given the OIB. After the first round, the judge’s decisions were presented to the group along with impact data. This was followed by further discussion before they moved on to the second round, where they were permitted to change the position of the bookmark. Results of both rounds are shown in table III 2.
TABLE III. Cut score results for bookmark method
Page number Ability ( ) after Mean θ Final cut Round 1 (number of Median adjusting for RP (SD) score judges) of .67 B1 8/9 (3) 11 (2.87) 10/11 -.13 logits 22 10/11 (2) 12/13 (2) 13/14 (1) 15/16 (1) B2 25/26 (1) 28.44 (1.42) 29/30 1.04 logits 29 28/29 (2) 29/30 (5) 30/31 (1) Round 2 B1 8/9 (7) 8.33 (.71) 8/9 -.70 logits 16 9/10 (1) 10/11 (1) B2 28/29 (6) 28.56 (.73) 28/29 .93 logits 28 29/30 (3)
Source: Created by the author
The median of the second set of bookmarks for all panellists is used as the group standard, and the lower value of theta should be taken in order to set the cut score (CoE, 2009). Here, the median corresponds to an ability level of θ = -.7 logits for B1 and θ = 0.93 logits for B2 once adjustments have been made for the RP of 0.67. This ability level is equivalent to raw scores of 16 for B1 and 28 for B2 when mapped onto the test characteristic curve.
(2) One panellist could not be present for the bookmark method and so this is the result for nine judges.
Revista de Educación, 381. July-September 2018, pp. 35-60 47 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
A final discussion then took place which considered both impact and external validity data. Participant judges were seen to be happy with the final decisions and in agreement that these should be used as the final cut scores for the test. The resulting impact data for the March 2017 administration give the final classifications of 135 (29%) fail, 166 (36%) CEFR B1 and 163 (35%) CEFR B2.
Procedural validity
The results from the post standard setting questionnaire are shown in table IV. For each question, participants answered a 4-point likert scale in which 1 represents complete agreement. Overall a very high level of agreement can be seen. The panellists felt confident about the procedure and their final decisions. Most notably, all panellists felt that the final cut score decisions were a fair and accurate representation of CEFR B1 and B2 for listening.
TABLE IV. Questionnaire results
Mean SD I felt I had a sound understanding of CEFR levels for listening after the familiarisa- 1.3 .48 tion sessions. The explanation of the test construct and specifications helped me. 1.5 .71 I understood the Basket method standard setting procedure. 1.2 .42 I felt confident about my decisions answering the question ‘At what CEFR level 1.6 .52 must test taker be in order to answer this item. I understood the concept of ‘minimally competent candidate’. 1 .00 The Basket method standard setting exercise helped me to understand test con- 1.2 .42 tent. I understood the Bookmark method standard setting procedure. 1.33 .71 I understood the concept of RP and what the 67% probability means. 1.11 .33 I felt confident about the placing of my bookmark on round 1. 1.67 .50 I felt confident about the placing of my bookmark on round 2. 1.11 .33 The final recommended cut scores of the group are a fair representation of CEFR 1 .00 B1 and B2 levels for listening.
Source: Created by the author
48 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
Internal validity
The internal validity of the standard setting procedure concerns the accuracy and consistency of results; the quality of judgments needs to be determined and reported. For the Basket method the results showed judges to be internally consistent with a mean Infit MNSQ of 0.98 and SD of 0.28. There is one overfitting judge and one underfitting judge shown by Infit MNSQ and Zstd. However, only one judge shows a slight underfit in terms of outfit and exact and expected observed agreements were very similar, as would be expected by raters acting as independent experts (Linacre, 2017). In an attempt to provide stronger CEFR linkage, the decisions made by the judges about item difficulty using the Basket method were also compared with the actual item difficulties provided by the Rasch analysis. This analysis shows further evidence of the quality of the standard setting judgements. The data set is small, with tied ranks for the judged items and a Kendall’s Tau correlation τ = .57, p < .001 shows a significant relationship between judged item difficulty (using median values from Round 2) and Rasch difficulty parameters. This result gives evidence that the judges recognized the increasing difficulty of the items. The classification accuracy of the cut scores needs to be evidenced. The final judged cut scores after Round 2 using the Bookmark method showed a high level of consensus with small SDs. The validity of these cut scores can be evidenced by reporting the standard error of judgement (SEJ), which is ‘an estimate of the likelihood of replicating the recommended cut scores’ (Tannenbaum & Cho, 2014, p.245). Cohen, Kane and Crooks (1999, p.364) argue that the SEJ should be ≤ 1⁄2 SEM. The results for this calculation for both rounds can be seen in table V.
TABLE V. Results for classification accuracy
Round 1 Round 2 B1 B2 B1 B2 Standard deviation of mean cut score (SD) 2.87 1.42 0.71 0.73 Standard error of test (SEM) 2.5 2.5 2.5 2.5
Standard error of judgement (SEJ) 1.01 0.50 0.25 0.26 0.4 0.2 0.1 0.1
Source: Created by the author
Revista de Educación, 381. July-September 2018, pp. 35-60 49 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
Thus, the SEJ was always smaller than one-half of SEM, and the cut scores fulfil the quality criterion. It should, however, be noted here that it has been argued that final round judgements do not usually have much variability because judges are encouraged to converge and come to a consensus decision (Linn, 2003).
External validity
External validity refers to the use and comparison of different standard setting methods, as well as comparisons or triangulation with measures from other studies. The present study includes two standard setting methods. As reported in previous studies, the two procedures produced different results for both the B1 and B2 cut scores. At B2, the Rasch score to measure table shows that there was a certain degree of discrepancy, with the difference at slightly more than two standard errors. At B1, however, the Basket cut score result falls within one standard error of the Bookmark result, giving evidence that the two standard setting methods produced quite consistent results on this part of the test, despite the Basket method being only used for content familiarisation. The results were further analysed by comparing the test scores of those candidates who had undertaken exam preparation courses at CLM with a predicted ability level provided by their teachers. Despite the small group of candidates in question, by checking the classification accuracy of these students, this comparison allowed for a certain amount of triangulation and further contributed to validity evidence. The results can be seen in table VI. This small-scale study based on a teacher judgement criterion shows that there is an extremely high correlation between teacher prediction and score on the test τ = .85, p < .001. There is a classification agreement of 100% for all actual predictions of B1 and B2 and this gives us some external validity evidence about the accuracy of the cut scores. It can be seen that of the candidates predicted by their teachers to be borderline at both A2/B1 and B1/B2 level, about 50% of candidates were granted the higher level. It should, however, be understood that the present study is small in scale; ideally, better external validity evidence would be gained from a much larger-scale study.
50 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
TABLE VI. Comparison of teacher predictions and cut score decisions
Predicted CEFR Level Listening Score on test CEFR level received on test Fail 1 Fail Fail 11 Fail A2/B1 12 Fail A2/B1 12 Fail A2/B1 14 Fail A2/B1 17 B1 A2/B1 19 B1 B1 17 B1 B1 19 B1 B1 19 B1 B1 22 B1 B1 22 B1 B1 23 B1 B1 25 B1 B1 26 B1 B1 27 B1 B1 27 B1 B1/B2 26 B1 B1/B2 27 B1 B1/B2 33 B2 B1/B2 33 B2 B1/B2 36 B2 B2 30 B2 B2 31 B2 B2 31 B2 B2 35 B2 B2 36 B2 C1 40 B2
Source: Created by the author
Revista de Educación, 381. July-September 2018, pp. 35-60 51 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
Discussion
This study has given evidence for a stronger claim on the interpretation and use of test scores in terms of CEFR alignment for the UGR B1/B2 test. The standard setting process has been described in detail as the core part of the linking process and, in doing so, it has been documented that a reasonable and systematic process was followed in order to reach the final standard. Good validity evidence has been provided to support the setting of recommended cut scores, in which the standard setting process may be considered a ‘blend of judgment, psychometrics, and practicality’ (Hambleton & Pitonoak, 2006, p.435), and where ‘the question is not whether the cut score is correct but whether decisions based on the cut scores are reasonable, broadly acceptable and have mostly positive consequences’ (Kane, 2017, p.11). The basis of the CEFR alignment manual (CoE, 2009) is without doubt the use of expert judgements, a methodology which is widely used and recommended in the standard setting literature. However, methodologies using expert judgement have been reported as unreliable, arguably due to the fact that the CEFR does not provide sufficient and precise descriptions of proficiency levels (Alderson et al., 2006; Fulcher, 2004; Weir, 2005). Judges in an alignment study may interpret CEFR descriptors differently or have their own internalized idea of just what it means to be at a CEFR level (Eckes, 2012; Harsch & Hartig, 2015; Papageorgiou, 2010). Indeed, North and Jones (2009, p.16) state, ‘..no amount of CEFR familiarisation and standardization, or estimation of indices of consistency and agreement, will prove that a given group of experts judging the level for a given language are not bringing their own culturally determined interpretation to the task’. Despite these limitations, however, the judges in the present study expressed satisfaction with their results and believe that the cut scores are representative of a candidate who can perform tasks associated with the level. The study has therefore provided further evidence towards the test’s substantive and construct validity. Both methodologies used required extensive discussion about matching individual item difficulty to the CEFR levels, and as such strengthens the claim that test content is CEFR-aligned, something which was also reported by Kanistra and Harsch (2017) in the Trinity ISE linking study. Indeed, the problem of defining a just-qualified candidate is a much easier task for atest
52 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR which has been developed to operationalize the CEFR model because the standard has already been built in, giving more meaning to the cut scores (Tannenbaum & Wylie, 2008). Conversely, a retro-fit type study would need to pay particular attention to the content specification stage of CEFR alignment in order to provide evidence that the content of the test measures the language skills described by the framework (Tannenbaum & Cho, 2014); various CEFR-linking studies have reported problems encountered when matching content to descriptors (e.g. Brunfaut & Harding, 2014). Furthermore, it has been argued that very few test providers pay attention to this stage of CEFR linkage (Green, 2017); if content is not aligned to the CEFR there is little justification for conducting a standard setting study. I would highlight here that the UGR test has been specifically developed to be representative of the CEFR; as such, the study is not a post-hoc linking study but forms part of the test development project itself, with the standard already implicitly built into the test. CEFR descriptors have been included in the test specifications and guidelines for item writers include a number of the categories presented in the CoE (2009) specification forms. This is similar to the a priori approach reported for the Pearson Test of English Academic (De Jong & Zheng, 2016). In terms of the methodology used in the study, there is no best method of standard setting; the methods should be matched to suit the situation. As has been noted, the main Bookmark study is considered to be appropriate for use in test situations based on the Rasch measurement model of test development. Furthermore, the Basket method study was found to be invaluable as a primer study in that it allowed judges to become familiar with the test content. Here it would have been almost impossible to begin directly with a Bookmark study, and this is especially true for a listening test. The OIB does not present the items in the same order as the original test; as such it would have been extremely difficult to deal with separate items without prior knowledge of test tasks, moving artificially through audio files in an individual and non-linear fashion to target the item in question. CoE (2009) recommends that for a listening test, judges should be provided with a computer in order to do this. However, it is the author’s firm belief that the methodology used in this study provides a far more pragmatic solution to this problem, and that any study using the Bookmark method for a listening test should first carry out a Basket method-type study. A similar approach was used by Harsch
Revista de Educación, 381. July-September 2018, pp. 35-60 53 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
and Hartig (2015), though their reasoning was different; they wanted to separate the content matching task from the item difficulty matching task because they were considered to be very separate judgment tasks. Furthermore, a review of studies using the bookmark method (Peterson, Schulz, & Engelhard, 2011) reported that judges feel confident about the resulting cut scores. Certainly, in the present study the cut scores are well supported by the external validity evidence. This evidence will need strengthening with future studies; larger data samples could be collected over time and perhaps a prototype group method, such as the one reported by Eckes (2012), could be employed. One other noteworthy comment was the fact that some items had an empirical difficulty value in the OIB which did not coincide with the judges’ opinions. This phenomenon has been found in previous studies (e.g. Figueras, Kaftandjieva, & Takala, 2013). Also, it has been widely reported that judges are often unable to correctly identify item difficulty (Alderson, 1993). The present study is not immune to this, as is evidenced by the divergence in the B2 cut score using the Basket versus the Bookmark method. However, it is the author’s belief that this difference may be explained by the fact that participants had subsequently received a lot more information when making their final decisions using the Bookmark method. Consequently, their decision was very much more informed and took into account actual item difficulty estimations as opposed to subjective, opinion-based estimations of difficulty only. For the B1 cut score, the judges placed the bookmark quite low on the continuum of difficulty due to the existence of items which they believed were more representative of high B1/B2 mastery. Nevertheless, after applying the RP adjustment, the final B1 cut score was very similar (within one standard error) using both methods of standard setting, further confirmation that the collection of multiple sources of evidence can increase confidence in qualitative decisions.
Conclusion
As regards the validity argument approach presented by ALTE, the UGR test has been developed following detailed test specifications based on the CEFR (observation inference). It shows scoring validity to the extent that all test tasks have been piloted and items show good
54 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR psychometric properties (evaluation inference). The present study has further strengthened the test’s extrapolation inference; cut scores can be considered to be both interpretable and consistent and the results can be used to set the ability level on all future versions of the test. Through the use of a system of common item equating (see Kolen & Brennan, 2014; North & Jones, 2009; Wright & Stone, 1979), all tasks on the UGR test can be placed on the same Rasch measurement scale; test difficulty will subsequently be the same on every version of the test, which adds support for the generalisation inference. Cut-scores have consequences not only for candidates but also for a whole range of other stakeholders, such as parents, educators, and educational policy makers. Standard setting is a fundamental part of the test development process; it should not therefore be treated simply as an isolated event but must instead be considered an essential and integral component of the continuing validation process (Papageorgiou & Tannenbaum, 2016). For the UGR test, this includes the implementation of future studies to monitor the consequences of applying the cut scores determined from the present study, which forms part of a whole range of validity checks implemented by the test developers. Similarly, a standard setting process has also been carried out for the UGR reading test, and similar studies are regularly carried out as part of the internal benchmarking and training sessions at the CLM for the speaking and writing parts of the test. Indeed, the validation of the interpretation and use of test scores for the UGR bi-level test is an ongoing process: it is just such an accumulation of validity evidence which provides the necessary backing for stakeholders to be confident of any decisions made based on test scores. In Spain, the CertAcles exams have gone some way to responding to the recommendations made by Halbach, Lafuente and Guerra (2013) concerning language accreditation and the unification of criteria. These tests are designed to be consistent with the CEFR and require that reliability and validity evidence be provided. However, as Deygers et al. (2017) have warned, the pressure for testing bodies to report CEFR alignment has lead to test misuse in many contexts where this claim is not substantiated. Here, I would urge all test providers in the current context to carry out standard setting studies so as to give greater meaning to their reported scores. While traditionally many institutions have viewed the cut score as a normative standard (e.g. 60%), test developers need to shift
Revista de Educación, 381. July-September 2018, pp. 35-60 55 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
that perspective towards cut scores that are defensible as a representation of the performance standard if we are to allow a strong claim for CEFR linkage of our tests.
References
AERA (American Educational Research Association), APA (American Psychological Association) & NCME (National Council on Measurement in Education). (2014). Standards for educational and psychological testing. Washington, DC: AERA. Alderson, J. C. (1993). Judgements in language testing. In D. Douglas & C. Chapelle (Eds.), A new decade of language testing research (pp. 46- 57). Alexandria, VA: TESOL. — (2012). Principles and Practice in Language Testing: Compliance or Conflict? Presentation at TEA SIG Conference: Innsbruck. Retrieved from http://tea.iatefl.org/inns.html Alderson, J.C., Figueras, N, Kuijper, H., Nold, G, Takala, S & Tardieu, C. (2006). Analysing Tests of Reading and Listening in Relation to the Common European Framework of Reference: The Experience of The Dutch CEFR Construct Project, Language Assessment Quarterly, 3(1), 3-30. ALTE/Council of Europe (2011) Manual for Language Test Development and Examining. For use with the CEFR. Retrieved from: http://www. coe.int/t/dg4/linguistic/ManualtLangageTest-Alte2011_EN.pdf Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly 2(1), 1–34. Bachman, L.F., & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press. Brunfaut, T., & Harding, L. (2014). Linking the GEPT listening test to the Common European Framework of Reference. Taiwan: Language Training and Testing Centre. Chapelle, C. A. (2012). Validity argument for language assessment: The framework is simple... Language Testing, 29(1), 19–27. Cizek, G. J. (2012). The forms and functions of evaluations in the standard setting process. In G. J. Cizek (Ed.), Setting performance standards:
56 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
nd Foundations, methods, and innovations (2 ed., pp. 165 - 178). New York: Routledge. — (2016). Validating test score meaning and defending test score use: Different aims, different methods. Assessment in Education: Principles, Policy & Practice, 23(2), 212-225. Cohen, A. S., Kane, M. T., & Crooks, T. J. (1999). A generalized examinee- centered method for setting standards on achievement tests. Applied Measurement in Education, 12(4), 343-366. Conferencia De Rectores De Las Universidades Españolas (CRUE). (2011, 8 de septiembre). Propuestas sobre la acreditación de idiomas. Retrieved from: http://www.acreditacion.crue.org/ Council of Europe. (2008). Recommendation CM/Rec (2008)7 of the Committee of Ministers to member states on the use of the Council of Europe’s Common European Framework of Reference for Languages (CEFR) and the promotion of plurilingualism. Strasbourg, France: Council of Europe. Retrieved from http://www.coe.int/t/dg4/ linguistic/Conventions_EN.asp — (2009). Relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment. Strasbourg, France: Council of Europe. Retrieved from: http://www. coe.int/T/DG4/Linguistic/Manuel1_EN.asp De Jong, J. & Zheng, Y. (2016) Linking to the CEFR: validation using a priori and a posteriori evidence. In, Banerjee, J. and Tsagari, D. (eds.) Contemporary Second Language Assessment. London, GB, Bloomsbury Academic pp. 83-100. (Contemporary Applied Linguistics, 4). Deygers, B., Zeidler, B., Vilcu, D., & Carlsen, C. H. (2017). One Framework to Unite Them All? Use of the CEFR in European University Entrance Policies. Language Assessment Quarterly, 1-13. Downey, N. & Kollias, C. (2010). Mapping the Advanced Level Certificate in English (ALCE™) examination onto the CEFR. Aligning Tests with the CEFR, Reflections on using the Council of Europe’s draft Manual, Martyniuk, W. (ed). Cambridge University Press. Cambridge. 119-129. Eckes, T. (2009). Many-facet Rasch measurement. In S. Takala (Ed.), Reference supplement to the manual for relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment (Section H). Strasbourg, France: Council of Europe/Language Policy Division.
Revista de Educación, 381. July-September 2018, pp. 35-60 57 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
— (2012). Examinee-centered standard setting for large-scale assessments: The prototype group method. Psychological Test and Assessment Modeling, 54, 257–283. Ferrara, S., & Lewis, D. (2012). The Item-Descriptor (ID) Matching method. In G. J. Cizek (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 255-282). New York: Routledge. Figueras, N. (2012). The impact of the CEFR. ELT journal, 66(4), 477-485. Figueras, N., Kaftandjieva, F., & Takala, S. (2013). Relating a Reading Comprehension Test to the CEFR Levels: A Case of Standard Setting in Practice with Focus on Judges and Items. Canadian Modern Language Review/La Revue Canadienne Des Langues Vivantes, 69(4), 359-385. Fulcher, G. (2004). “Deluded by artifices? The Common European Framework and harmonization.” Language Assessment Quarterly, 1(4), 253 - 266. Fulcher, G., & Owen, N. (2016). Dealing with the demands of language testing and assessment. The Routledge Handbook of English Language Teaching, 109-120. Green, A. (2017). Linking Tests of English for Academic Purposes to the CEFR: The Score User’s Perspective, Language Assessment Quarterly, 1-16. Halbach, A., Lazaro Lafuente, A., & Perez Guerra, J. (2013). La lengua inglesa en la nueva universidad española del EEES: The role of the English language in post-Bologna Spanish universities. Revista de Educación, 362, 105-132. Hambleton, R., & Jirka, S. (2006) Anchor-based methods for judgmentally estimating item statistics. In S. Downing and T. Haladyna (Eds.), Handbook of test development (pp. 399-420). Mahwah, Nj: Erlbaum. Hambleton, R., & Pitoniak, M. J. (2006). Setting performance standards. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 433–470). Westport, CT: American Council on Education/ Praeger Publishers. Harsch, C. & Hartig, J. (2015). What Are We Aligning Tests to When We Report Test Alignment to the CEFR?, Language Assessment Quarterly, 12:4, 333-362. Kaftandijeva, F. (2010). Methods for setting cut scores in criterion- referenced achievement tests. A comparative analysis of six recent methods with an application to tests of reading in EFL. Cito, Arnhem: EALTA. Retrieved from: www.ealta.eu.org/documentsresources/FK_ second_doctorate.pdf
58 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
Kane, M. (2012). Validating score interpretations and uses. Language Testing, 29 (1), 3 –17. — (2017). Using Empirical Results to Validate Performance Standards BT - Standard Setting in Education: The Nordic Countries in an International Perspective. In S. Blömeke & J.-E. Gustafsson (Eds.), (pp. 11–29). Cham: Springer International Publishing. Kanistra, V. & Harsch, C. (2017). Using the Item Descriptor Matching method to enhance validity when aligning test to the CEFR. 4th Meeting of the EALTA CEFR Special Interest Group. Retrieved from: http://www.ealta.eu.org/ events/SIG_CEFR_london2017/presentations/Presentations/1400-1530/ CEFR%20SIG%20Voula%20and%20Claudia.pdf Kantarciouglu, E, Thomos, C, O’Dwyer, J & O’Sullivan, B. (2010) ‘Benchmarking a high-stakes proficiency exam: the COPE linking project’, in W. Martyniuk (ed)Aligning Tests with the CEFR: reflections on using the Council of Europe’s draft Manual. Cambridge University Press. Cambridge. Kolen, M. J., & Brennan, R. L. (2014). Test equating, scaling, and linking: methods and practices (3rd ed.). New York: Springer-Verlag. Linacre, J. M. (2017). Facets computer program for many-facet Rasch measurement, version 3.80.0. Beaverton, Oregon: Winsteps.com Linn, R. L. (2003). Performance Standards: Utility for Different Uses of Assessments. Education policy analysis archives, 11, 31. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: Macmillan. Mitzel, H.C., Lewis, D.M., Patz, R.J., & Green, D.R. (2001). The bookmark procedure: Psychological perspectives. In G.J. Cizek (Ed), Setting performance standards: Concepts, methods and perspectives (pp. 249- 281). Mahwah, NJ: Lawrence Erlbaum Assoc. North, B., & Jones, N. (2009). Relating language examinations to the Common European Framework of Reference for Languages. Further material on maintaining standards across languages, contexts and administrations by exploiting teacher judgment and IRT scaling. Strasbourg: Language Policy Division. Papageorgiou, S. (2010). Investigating the decision-making process of standard setting participants. Language Testing, 27(2), 261–282. Papageorgiou, S., & Tannenbaum, R. J. (2016). Situating Standard Setting within Argument-Based Validity. Language Assessment Quarterly, 13(2), 109–123.
Revista de Educación, 381. July-September 2018, pp. 35-60 59 Received: 11-05-2017 Accepted: 02-02-2018 Shackleton, C.. Linking the University of Granada CertAcles listening test to the CEFR
Peterson, C. H., Schulz, E. M. and Engelhard Jr., G. (2011), Reliability and Validity of Bookmark-Based Methods for Standard Setting: Comparisons to Angoff-Based Methods in the National Assessment of Educational Progress. Educational Measurement: Issues and Practice, 30: 3–14. Reckase, M. D. (2010). NCME 2009 presidential address: What I think I know. Educational Measurement: Issues and Practice, 29(3), 3–7. Tannenbaum, R, J. & Cho, Y. (2014). Critical Factors to Consider in Evaluating Standard-Setting Studies to Map Language Test Scores to Frameworks of Language Proficiency,Language Assessment Quarterly, 11(3), 233-249. Tannenbaum, R, J. & Wiley E. C. (2008). Linking English-language test scores onto the Common European Framework of Reference: An application of standard-setting methodology. ETS Research Report Series, 2008(1). Weir, C. J. (2005). Limitations of the council of Europe’s framework of reference (CEFR) in developing comparable examinations and tests. Language Testing, 22(3), 281–300. Wright, B. D., & Stone, M. H. (1979). Best test design. Chicago: MESA.
Contact address: Caroline Shackleton. Centro de Lenguas Modernas de la Universidad de Granada. Placeta del Hospicio Viejo s/n 18009, Granada, Spain. E-mail:[email protected]
60 Revista de Educación, 381. July-September 2018, pp. 35-60 Received: 11-05-2017 Accepted: 02-02-2018 Impact of the economic crisis on school segregation in Spain1
Incidencia de la crisis económica en la segregación escolar en España
DOI: 10.4438/1988-592X-RE-2017-381-381
F. Javier Murillo Cynthia Martínez-Garrido Universidad Autónoma de Madrid
Abstract The equity of the education system, in particular, the magnitude of school segregation has been affected by the severe economic crisis in Spain since 2008? The aim of this study is to determine the evolution across the last 15 years of the socioeconomic school segregation and immigrant school segregation in Spain and its Autonomous Communities. To achieve our aim, we conduct a special exploitation of the available PISA dataset from 2000 to 2015; we estimated four segregation indexes: Gorard index, Dissimilarity index, Hutchens index, and Isolation index. The socioeconomic school segregation considerer as a minority group the 25% of students from families with lower socioeconomic and cultural level (Q1), as well as the 25% with higher socioeconomic and cultural level (Q4). The minority group considered to estimate the immigrant school segregation was the students born abroad. According to our data, the socioeconomic school segregation in Spain has slightly decreased from 2000 to 2012, and increased strongly to 2015 (3.1% for students from Q1, and 3.6% students from Q4). Along the increases of the percentage of immigrant students from 2000 to 2009, the immigrant school segregation has decreased. From this moment, the immigrant school segregation has remained, even though the percentage of immigrant
(1) This paper has been developed within the framework of the R+D+i Excellence Project “Schools in socio-economically challenging contexts: An approach from education for social justice research” Ref: EDU2014-56118-P. Funded by the Ministry of Economy and Competitiveness-Government of Spain.
Revista de Educación, 381. July-September 2018, pp. 61-87 61 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
students keep growing. These results, let us confirm that there is evidence of the impact that economic crisis has into the increase of the school segregation in Spain. This impact stemming the decrease trend since 2000. A greater effort must be made by the educational administrations to terminate this state of affairs. They should strive to pursue the decreased trend that already existed before economic crisis came. The differences founded between the Autonomous Communities let us know that the public education policy can help to develop an equitable educational system, or it can serve to heighten social inequalities.
Keywords: School, Evolution, Equity, Economic recession, Immigration, Economic situation, School integration.
Resumen ¿La grave crisis económica que afecta a España desde 2008 ha tenido su repercusión en la equidad del sistema educativo, concretamente en la magnitud de la segregación escolar? Este estudio tiene por objetivo determinar la evolución en los últimos 15 años de la segregación escolar de carácter socioeconómico y por origen nacional en España y las Comunidades Autónomas. Para ello, se realiza una explotación especial de los datos disponibles del estudio PISA desde el año 2000 hasta 2015 y se estiman cuatro índices de segregación: índice de Gorard, de Disimilitud, de Hutchens y de Aislamiento. La segregación socioeconómica se calcula para el 25% de estudiantes con familias de menor socioeconómico y cultural (Q1) y el 25% mayor (Q4), y la segregación por origen nacional se estima considerando como grupo minoritario a los estudiantes nacidos en el extranjero. Los resultados indican que la segregación escolar por nivel socioeconómico ha disminuido ligeramente desde 2000 hasta 2012, y ha aumentado fuertemente hasta 2015 (un 3,1% para Q1 y 3,6% Q4). La segregación escolar por origen nacional, acompañada por el aumento del porcentaje de extranjeros, ha disminuido desde el 2000 de forma importante hasta 2009. Entonces se ha producido un estancamiento a pesar del crecimiento del porcentaje de inmigrantes. De esta forma, se puede afirmar que hay evidencias de que la crisis ha incidido en el aumento de la segregación escolar en España, rompiendo la tendencia a la disminución desde 2000. Urge, por tanto, que las Administraciones educativas tomen medidas que reviertan esa situación y que encaminen al sistema educativo en esa tendencia a la baja que había antes de la crisis. Las diferencias encontradas por Comunidades indican que la política educativa pública puede profundizar en la construcción de un sistema educativo equitativo o, por el contrario, que contribuya a profundizar las desigualdades sociales.
Palabras clave: Escuela, Evolución, Equidad, Recesión económica, Inmigración, Condiciones económicas, Integración escolar.
62 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Introduction
In 2008 Spain was thrown into a financial crisis that affected –and continues to affect– the lives of millions. The education system is undoubtedly one of the social institutions that has most suffered from the recession and the decisions that have been made in its aftermath. The significant restrictions made to budgets in this area, including 13% of public investment in education (Mas Ivars, Pérez García and Uriel, 2015) have had serious consequences. To cite a few examples, since 2009 and despite the increase in students registered, there are between 28,000- 32,000 fewer teachers, the economic and working conditions of teachers have worsened, and student ratios in the classroom have increased, affecting public schools five times more than the private education system. The crisis has also had an impact on families too, however, especially in the homes of boys and girls whose parents’ sources of income have dwindled significantly, so much so that it is estimated that currently one in three children in Spain is at risk of poverty and social exclusion (Ayllón, 2015). This, in turn, has strong implications for the education system (García-Gómez and Cabanillas, 2017). There can be no doubt that in order for society to be more cohesive and just, we need an education system that shares these very characteristics –although admittedly this, on its own, would not be enough. School segregation is an aspect of educational inequality that threatens equality in terms of opportunities and has a direct impact on the generation of social inequality. Has the financial crisis affected school segregation in Spain? We do not know. We have some evidence of school segregation at certain points in time (e.g. Ferrer, 2008; Mancebón-Torrubia and Pérez- Ximénez, 2009; Murillo and Martínez-Garrido, 2018), but until now no evolutionary study has been carried out that allows us to analyse what has happened in recent years. In view of the above, the aim of this study is to identify the development of school segregation depending on socioeconomic level and national origin of the students in Spain and the autonomous communities, using data from the international PISA assessment, in order to determine the impact of the financial crisis.
Revista de Educación, 381. July-September 2018, pp. 61-87 63 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Previous studies
School segregation is understood as the unequal distribution of students in schools depending on their personal and social characteristics, or their condition. As Benito and González-Baetbó (2007, p. 41) suggest, school segregation spawns a “situation of social homogeneity within schools and social heterogeneity between schools”. The first studies on school segregation checked the implementation of measures following the so-called Brown case, through which, on 17th May 1954, the Supreme Court in the USA declared that the segregation based on skin colour in North American schools was unconstitutional. In the second half of the 50s, a fruitful line of research opened up that aimed to identify the extent of ethnic school segregation (Orfield, Ee, Frankenberg and Siegel-Hawley, 2016; Reardon and Owens, 2014; Stroub and Richards, 2013). This concern for the unequal distribution of students turned into the study of segregation according to student national origin (e.g. Cebolla-Boado and Garrido Medina, 2011; Stephan, 2013; Van-Zanten, 2007), and in the 80s, to the study of school segregation depending on socioeconomic status (e.g. Gorard, Hordosy and See, 2013; Owens, Reardon and Jencks 2016) In Spain, research into the extent of school segregation is recent, starting in the past decade only, and focuses on both school segregation depending on economic, social and cultural status and segregation by national origin. Table I offers a general overview of the most important research carried out to date.
TABLE I. Main studies on the extent of school segregation in Spain
STUDY SEGREGATION SAMPLE DATABASE INDEX Benito and González- Parents’ educational Municipalities Generalitat of G, P Baetbó (2007) background in Catalonia Catalunya 2004 National origin Sánchez Hugalde (2007) National origin Municipalities Generalitat of D in Catalonia Catalunya 2002 Alegre, Benito and National origin Municipalities Generalitat of G González-Baetbó (2008) in Catalonia Catalunya 2005
64 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Ferrer (2008) Socioeconomic Spain and PISA 2003 D and cultural level Catalonia Parents’ educational background Síndic (2008) National origin Catalonia Generalitat of Ca- D, A, H talunya from 2001 to 2006 Valiente (2008) National origin Catalonia Generalitat of D, A, H Catalunya 2001 and 2006 Mancebón-Torrubia and Parents’ educational Spain PISA 2006 D, C Pérez-Ximénez (2009) background National origin Mancebón-Torrubia and Socioeconomic and Spain PISA 2006 D Pérez-Ximénez (2010) cultural level Ferrer, Castejón, Castel Socioeconomic and Spain and PISA 2009 D and Zancajo (2011) cultural level Autonomous Communities Bonal (2012a, b) National origin Municipalities Generalitat of D in Catalonia Catalunya 2010 Murillo, Martínez-Garrido National origin Spain and PISA 2015 G and Belavi (2017) Autonomous Communities Murillo and Martínez- Socioeconomic and Spain and PISA 2015 G, A Garrido (2018) cultural level Autonomous Communities
Note: A: Isolation index. C: Contact index. D: Dissimilarity index. G: Gorard index. H: Hutchens index. P: Polarization index. Source: Elaborated by the authors.
Research that has aimed to assess the extent of school segregation in Spain has tackled three types of segregation. Firstly, a considerable number of studies deal with school segregation depending on economic, social and cultural status, that is, the unequal distribution of students in schools depending on the economic, social and cultural status of their families. Such studies include those by Ferrer (2008), Ferrer et al. (2011), Mancebón-Torrubia and Pérez-Ximénez (2010) and Murillo and Martínez- Garrido (2018). Other studies look at the different distribution of boys and girls in schools depending on their parents’ educational background (Benito and González-Baetbó, 2007; Ferrer, 2008; Mancebón-Torrubia and Pérez-Ximénez, 2009). A final group of studies, and the largest in number, focus on school segregation depending on pupil nationality, that is, on the possible concentration in some schools of students who
Revista de Educación, 381. July-September 2018, pp. 61-87 65 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
were not born in Spain (Alegre et al., 2008; Benito and González-Baetbó, 2007; Bonal, 2012a, 2012b; Mancebón-Torrubia and Pérez-Ximénez, 2009; Murillo at al., 2017; Sánchez Hugalde, 2007; Síndic, 2008; Valiente 2008). The study of the extent of segregation demands access to large statistically representative samples. Perhaps for this reason studies have looked at Spain as a whole and, depending on the availability of data, on Spain’s autonomous communities using the different editions of the PISA study (2003, 2006, 2009 and 2015, or, when it comes to Catalonia, using the data from the region’s Department of Education (Departament d’Educació de la Generalitat de Catalunya). Our only hope is that the current obsession for assessment will onto provide the scientific community with data for research that allows us to better understand the Spanish education system. There is no consensus, however, between Spanish researchers when it comes to which index to use to estimate the magnitude of school segregation. The most popular index is the Dissimilarity index (Duncan and Duncan 1955), which reflects the proportions of students belonging to the minority group that would need to change school for there to be an equal distribution between schools. Other indexes used include the Gorard index (2009), a variation of the Dissimilarity index that controls the size of the minority group and that is interpreted as the number of students that would need to change school for equal distribution to be achieved within the geographical area analysed. The Hutchens index (2004) is also used, which measures how far each school is from equal distribution, that is, the distance between the geometric average of those members of the minority group taking part in the absence of the segregation and geometric average of the other students. The Polarization index proposed by Benito and González-Baetbó (2007) is less used, and estimates the distribution of a subgroup of students between 20% of schools with the greater and lesser proportions of them in a particular area. These five indexes are used to measure what is refer to asthe dimension of Uniformity of segregation: the degree of imbalance with which the members of the minority group are distributed between the schools. The other dimension of school segregation is that of Exposure, that is, the probability of coming across someone from the same group in the same school. To look into this, studies in Spain use the Isolation index (Lieberson 1981) or, less often, the Contact index (Coleman, Hoffer and Kilgore, 1982). The problem that arises from the disagreement over
66 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain which index to use is that each index has its own meaning and, although they are all closely related (Murillo, 2016), some give higher ratings than other. As a result, their estimations are not comparable. Of the four most used indexes, the Dissimilarity index gives the highest readings, followed by the Gorard index and, finally, the Hutchens index. In addition to measuring another dimension, the Isolation index’s results also stand out because of the size of the minority group studied. We should add here that in order to measure school segregation the minority group studied must be defined, since the results vary depending on the one chosen to look at. Analysing the findings of these studies, we have discovered that school segregation based on socioeconomic status in Spain according to PISA 2003 was 0.40, taking as a minority group the 25% of students with a lower socioeconomic status (Ferrer, 2008). Using PISA 2006 data, Mancebón-Torrubia and Pérez-Ximénez (2010) estimate this figure at 0.39, using as a minority group the 25% of students with a higher level of income –values measured using the Dissimilarity index. More recent results are those published by Murillo and Martínez-Garrido (2018) using data from PISA 2015, who estimate segregation to be 0.38 (Gorard index) and 0.32 (Isolation index), averaging the segregation of 25% of students with higher and lower socioeconomic statuses. In Catalonia, Benito and González-Baetbó (2007) highlighted that segregation in 2004 varied between 0.20 and 0.66, depending on the area studied (values measured using the Gorard index). School segregation depending on parents’ educational background in Spain according to Ferrer (2008) using PISA 2003 is 0.34, using as a minority group those students whose parents had no qualifications or did not finish primary education and using the rest as a majority group. Using PISA 2006, Mancebón-Torrubia and Pérez-Ximénez (2009) found segregation to be at 0.29; in this case, the minority group was made up of students whose parents had no qualifications or only had completed primary education. The majority group was made up of students whose parents had higher levels of qualifications than basic ones. In Catalonia, segregation can vary from 0.45 for those students whose parents had no qualifications or had not finished compulsory education, to 0.37 if the parents had been to university, according to Benito and González-Baetbó (2007). These were average values from the municipalities that took part in the study and were measured using the Dissimilarity index.
Revista de Educación, 381. July-September 2018, pp. 61-87 67 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Finally, segregation in public schools of immigrant students in Spain was analysed by Mancebón-Torrubia and Pérez-Ximénez (2009) using PISA data. In 2006, authors noted that the segregation in public schools was 0.53, or 0.57 if all schools were taken into account (values were measured using the Dissimilarity index). Also, using PISA data, this time from 2015, Murillo et al. (2017) demonstrate that school segregation of students who were born abroad was 0.45 in Spain (according to the Gorard index). In Catalonia, Valiente (2008) estimates that the segregation value for 2001 was 0.55 (Dissimilarity index) and 0.09 (Isolation index). On the other hand, Síndic (2008) estimates the value at 0.36 using PISA data from 2006 (Dissimilarity index). As we have seen in our revision of previous studies, research conducted to date only offers some evidence of school segregation at certain points in time. For this reason, the present study aims to ascertain how school segregation has evolved depending on students’ socioeconomic status and their national origin in Spain and its autonomous communities, so that we have evidence of the impact of the financial crisis. To this end, we make extensive use of all of the data available in PISA from 2000-2015 (OCDE, 2016).
Method
This study analysed data from the editions of PISA from 2000, 2003, 2006, 2009, 2012 and 2015 –all the editions available to date. Since 2000 the PISA programme has studied every three years the results of the educational systems of different countries in three key areas: Mathematics, Language and Science. It extracts information from factors associated with learning that allows us to contextualise and, to an extent, explain the results obtained. To gather data, PISA uses some performance tests, context questionnaires that are sent to the school principals, the families and students of Secondary Education. The variable used to measure school segregation was the Index of Economic, Social and Cultural Status (IESC), as defined by PISA. This index is made up of partial indexes related to each pupil’s economic, social and cultural background (index of the greater level of education of his/her parents, index of the greater level of employment of the parents and index of family’s cultural heritage). The IESC index has an average
68 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain value of zero and a typical deviation of one for all of the countries taking part in PISA. This means that a negative mark indicates that the families of students in one of the countries has an economic, social and cultural level that is inferior to the average of the countries taking part in the PISA programme (OCDE, 2016).
Sample
In order to select the sample, PISA uses a two-staged stratified sampling design, with samples in the first stage proportional to size. The primary sampling units are the schools and the secondary ones are the students in each school: 35 students were chosen at random from lists of the subjects offered at each school. The explicit stratification variables used for Spain were the form of control (public/private), its autonomous communities, the size of the school and, in the case of the Basque Country, the mode of education. The implicit stratification variables used were postcodes and, in Catalonia, the size of the area studied. The effective selection of the sample was carried out using a systematic approach with a sampling interval. The distribution of students and schools in the sample can be seen in table II.
TABLE II. Sample. Number of students, schools and % of immigrants
2000 2003 2006 % % % Stud. Schools Stud. Schools Stud. Schools Inmig Inmig Inmig Andalusia 1,463 51 2.10 Aragon 1,526 51 4.95 Asturias 1,579 51 2.09 Balearic Islands Canary Islands Cantabria 1,496 53 3.56 Castile and 1,490 51 1.46 1,512 52 2.74 León Cast.-La Mancha
Revista de Educación, 381. July-September 2018, pp. 61-87 69 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Catalonia 1,515 50 3.47 1,527 51 7.91 Extremadura Galicia 1,573 53 1.54 La Rioja 1,333 45 5.81 C. of Madrid Reg. of Murcia Navarre 1,590 52 6.93 Basque 3,885 141 1.48 3,929 151 3.49 Country Valencian C. Total Spain 6,135 185 2.54 1,790 383 2.83 19,604 684 6.10
2009 2012 2015 % % % Stud. Schools Stud. Schools Stud. Schools Inmig Inmig Inmig Andalusia 1,416 51 5.20 1,434 52 3.06 1,813 54 3.23 Aragon 1,514 52 11.52 1,393 49 12.36 1,798 53 12.45 Asturias 1,536 54 4.59 1,611 55 5.86 1,790 54 5.79 Balearic 1,463 52 13.00 1,435 54 16.60 1,797 54 14.24 Islands Canary 1,448 50 9.62 1,842 54 9.60 Islands Cantabria 1,516 51 6.57 1,523 54 8.99 1,924 56 9.00 Castile and 1,515 51 4.86 1,592 55 6.10 1,858 57 6.60 León Cast.-La 1,889 55 7.63 Mancha Catalonia 1,381 50 9.49 1,435 51 11.12 1,769 52 14.81 Extremadura 1,536 53 2.86 1,809 53 1.74 Galicia 1,585 54 3.30 1,542 56 4.50 1,865 59 5.07 La Rioja 1,288 46 12.05 1,532 49 16.76 1,461 47 15.70 C. of Madrid 1,453 51 14.55 1,542 51 13.31 1,808 51 14.84 Reg. of 1,321 51 11.49 1,374 51 14.85 1,796 53 12.77 Murcia Navarre 1,504 49 12.04 1,530 51 14.56 1,874 52 13.10 Basque 4,768 177 4.35 4,738 174 8.17 3,612 119 7.75 Country Valencian C. 1,370 21 1,625 53 12.28 Total Spain 25,887 889 8.42 25,312 893 7.85 6,736 201 9.07
Note: the percentage of immigrant is weighted. Source: Elaborated by the authors.
70 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Procedure
In order to estimate the extent of school segregation in the whole of Spain, for segregation based on both socioeconomic status and national origin, four different segregation indexes were used: the Godard index, the Dissimilarity index, the Hutchens index and the Isolation index. The first three of these look at the uniformity of school segregation, and the final one examines exposure (Murillo, 2016). The reason behind using so many strategies it to provide information that is comparable with that of similar studies, and to prove the stability between the different results, thus confirming their trustworthiness. Due to space limitations in this article, we cannot show the results of the four indexes in our analysis of the evolution region by region. Instead, we have chosen to use the Gorard index for this, since it is the only index in which the results remain the same despite the size of the minority group. This makes this index particularly attractive when studying the evolution of school segregation (Gorard, 2009; Gorard and Taylor, 2002). See Murillo (2016) for an analysis of the indexes. Each year the data is available for, it was estimated the socioeconomic school segregation the 25% of students from families of a lower economic, social and cultural status (Q1) and for the 25% of students of a higher economic, social and cultural status (Q4). This double perspective gives us a more comprehensive idea than if only Q1 is used (as is usual in this kind of study). In order to assess school segregation according to the national origin of the students, the minority group is first-generation immigrant students, that is, those students who were not born in Spain.
Results
According to the aims of the study, first we present the results of our analysis of the development of school segregation according to socioeconomic status, to then look at segregation by national origin. In both cases, we offer an overall vision of Spain as a whole and then a look at each of the 17 autonomous communities independently.
Revista de Educación, 381. July-September 2018, pp. 61-87 71 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
The evolution of school segregation according to socioeconomic status
School segregation according to socioeconomic status in Secondary Schools in Spain declined slightly from 2000-2012 before increasing drastically (graph I). This is clear independently from the index, and both if we consider the 25% of students from families with a lower economic, social and cultural status (Q1) and the segregation of the 25% of students from families of a higher economic, social and cultural status (table 3).
GRAPH I. Evolution of school segregation depending on socioeconomic status in Compulsory Secondary Education in Spain. Gorard index for the 25% of students from families with a lower economic, social and cultural status (Q1) and the 25% with a higher economic, social and cultural status (Q4)
Source: Elaborated by the authors by PISA 2000, 2003, 2006, 2009, 2012 and 2015.
Indeed, if we look at the extent of segregation according to the Gorard index for the 25% of students from families of a lower economic, social and cultural status for the whole of Spain, this drops from 0.30 in 2000 to 0.28 in 2012. It then increases to 0.31. Analysing segregation using any of the other indexes offers us a similar picture. With the Dissimilarity index, for example, it has dropped
72 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain by 0.03 points from 2000 to 2012 (table III). In 2015, however, this figure rises again to 0.42, topping the numbers reached in 2000. The Hutchens index moves from 0.16 in 2000 to 0.14 in 2012 and from there to 0.17. Finally, using the Isolation index, from 0.39 it drops to 0.37 in 2012 and then goes back up to 0.39 in 2015. Bearing in mind that the first three indexes examine segregation in terms of uniformity (although with different hypotheses and results) and the Isolation index measures the dimension of exposure, such similar behaviour shows that the results are trustworthy. This data could provide us with a tentative and provisional answer to the question behind our very research. One plausible reason behind the significant increase in segregation between 2012 and 2015 could be the financial crisis that Spain has suffered since 2008. We will come back to this idea in our conclusions.
TABLE III. Evolution of school segregation according to socioeconomic status for the 25% of students with families with a lower economic, social and cultural status (Q1)
2000 2003 2006 2009 2012 2015 Gorard index 0.3023 0.3036 0.2997 0.2963 0.2806 0.3113 Dissimilarity index 0.4030 0.4045 0.3991 0.3945 0.3732 0.4152 Square Root index 0.1617 0.1612 0.1563 0.1582 0.1434 0.1734 Isolation index 0.3854 0.3898 0.3823 0.3788 0.3699 0.3892
Source: Elaborated by the authors by PISA 2000, 2003, 2006, 2009, 2012 and 2015.
The extent of segregation for the 25% of students from families of a greater economic, social and cultural status supports our findings further (Q4). To start with, school segregation in Spain has continued at significantly higher levels for students with Q4 families than for Q1 families. In this case, the lessening of the levels of segregation between 2000 and 2012 is not lineal but declines between 2000-2003 to then increase until 2009, when a slight drop occurred. The increase between the years 2012-2015, in this case, is much more marked for Q4 children than for Q1 children. Indeed, for the four indexes used, the decrease between 2000 and 2012 is 4.8% for the Hutchens index, 4% for the Isolation index, 2.6% for the Dissimilarity index and 2.1% for the Gorard index. In 2015,
Revista de Educación, 381. July-September 2018, pp. 61-87 73 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
the amount by which the numbers dropped was recovered and in just 3 years’ segregation rose 3.6% and 4.69% depending on the index used (table IV). In this way, we can anticipate here a second conclusion regarding the possible impact that the financial crisis may have had on segregation in schools. The data indicates that the crisis may well be affecting a greater concentration of students depending on the economic, social and cultural status of their families, but this concentration is greater between students with families of a higher economic, social and cultural status than those from families of a lower economic, social and cultural status.
TABLE IV. Evolution of school segregation according to socioeconomic status in Spain for the 25% of students from families of a higher economic, social and cultural status (Q4)
2000 2003 2006 2009 2012 2015 Gorard index 0.3387 0.3116 0.3164 0.3281 0.3182 0.3542 Dissimilarity index 0.4514 0.4156 0.4227 0.4377 0.4253 0.4722 Square Root index 0.1970 0.1528 0.1675 0.1707 0.1552 0.1954 Isolation index 0.4493 0.4031 0.4186 0.4322 0.4097 0.4459
Source: Elaborated by the authors by PISA 2000, 2003, 2006, 2009, 2012 and 2015.
The detailed analysis of the evolution of school segregation according to economic, social and cultural status in each autonomous community gives us an insight into the characteristics of each and the different measures taken in the 15 years studied. Starting with the analysis of each region for the 25% of students with families of a lower economic, social and cultural status (table V), we can see that the phenomenon seen generally throughout Spain as a whole is not the same in all of the regions:
■ In Andalucía, the Community of Madrid and Catalonia we see a practically uninterrupted increase in the segregation from when the first data is recorded (2003, 2006 and 2009) until 2015. ■ In Castile and León and Galicia, on the other hand, we see a decrease in segregation over the years studied, although both regions have some peaks and troughs in their results.
74 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
■ In La Rioja, the segregation recorded is the same for the whole of the country: it drops until 2012 and then rises drastically. ■ In Aragon and the Balearic Islands we see the opposite happen to the rest of Spain: it goes up until 2012 and from that point on it suddenly drops.
In the rest of the country, no clear tendency can be pinpointed (Asturias, Cantabria, Murcia and the Basque Country) or there is not enough data available for us to be able to draw conclusions (Canary Islands, Castilla-La Mancha, Extremadura and Valencian Community).
TABLE V. Evolution of school segregation according to socioeconomic status in Compulsory Secondary Education in Spain. Gorard index for the 25% of students from families with a lower economic, social and cultural status (Q1)
2000 2003 2006 2009 2012 2015 Andalusia 0.2578 0.2599 0.2599 0.2606 Aragon 0.2452 0.2608 0.2991 0.2221 Asturias 0.2420 0.2775 0.2534 0.2636 Balearic Islands 0.2489 0.2564 0.2019 Canary Islands 0.2681 0.2569 Cantabria 0.2334 0.2528 0.1998 0.2230 Castile and León 0.2633 0.2366 0.2799 0.2493 0.2274 Cast.-La Mancha 0.2623 Catalonia 0.2824 0.2778 0.2979 0.2946 0.3047 Extremadura 0.2609 0.2254 Galicia 0.2875 0.2742 0.2900 0.2294 La Rioja 0.2552 0.2290 0.2189 0.2712 C. of Madrid 0.2780 0.3326 0.3632 Reg. of Murcia 0.2745 0.2219 0.2438 Navarre 0.2701 0.2535 0.2467 0.2525 Basque Country 0.2752 0.2834 0.2732 0.2770 0.2678 Valencian C. 0.2234 Spain 0.3023 0.3036 0.2997 0.2963 0.2806 0.3113
Source: Elaborated by the authors by PISA 2000, 2003, 2006, 2009, 2012 and 2015.
Revista de Educación, 381. July-September 2018, pp. 61-87 75 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
A detailed analysis of school segregation in the different regions for the 25% of students from families of a greater economic, social and cultural status highlights a range of aspects that give us an insight into the different realities of each region, as a result of the different education policies in place, amongst other things. As a sample, we take 4 representative regions, starting with the Community of Madrid, which has the questionable honour of being the most segregated region in Spain. After dropping between 2009 and 2012, Madrid’s rate rises in 2015, undoubtedly because of the financial crisis but also due to the policies put in place in recent years. Catalonia, on the other hand, after keeping its rates more or less stable between 2003 and 2012, then goes on to increase drastically from then onwards. Between 2003 and 2005, the Basque Country managed to lower its levels of school segregation significantly year on year. Finally, in Andalusia, we see no clear tendencies in the 9 years covered in our study (table VI).
TABLE VI. Evolution of school segregation according to socioeconomic levels in Compulsory Secondary Education in Spain. Gorard index for the 25% of students from families of a greater economic, social and cultural status (Q4)
2000 2003 2006 2009 2012 2015 Andalusia 0.2970 0.2893 0.3040 0.2885 Aragon 0.2933 0.3113 0.3122 0.2569 Asturias 0.3320 0.3097 0.3215 0.3392 Balearic Islands 0.3191 0.2375 0.2396 Canary Islands 0.2747 0.3447 Cantabria 0.2915 0.2951 0.2600 0.2775 Castile and León 0.3023 0.2412 0.2886 0.2756 0.2831 Cast.-La Mancha 0.3443 Catalonia 0.3008 0.3362 0.2898 0.3088 0.3466 Extremadura 0.2829 0.2689 Galicia 0.3325 0.2986 0.2975 0.2647 La Rioja 0.2672 0.2778 0.2777 0.2505 C. of Madrid 0.4027 0.3751 0.3925 Reg. of Murcia 0.3091 0.2727 0.2843 Navarre 0.3044 0.2916 0.3152 0.2931 Basque Country 0.3131 0.3129 0.2887 0.2825 0.2794 Valencian C. 0.2793 Spain 0.3387 0.3116 0.3164 0.3282 0.3182 0.3542
Source: Elaborated by the authors by PISA 2000, 2003, 2006, 2009, 2012 and 2015.
76 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Evolution of School Segregation by National Origin
A look at the distributions of students in schools depending on whether or not they were born in Spain or abroad (first-generation immigrants) can also give us more information on this aspect. Bearing in mind the close relationship that exists between the proportion of students from the minority group and the segregation itself (Murillo 2016), graph II shows us the evolution of the extent of segregation and the percentage of foreign students registered. The results show that the percentage of immigrant students has seen a practically uninterrupted growth from the year 2000 (2.54%) to 2015 (9.07%) and, at the same time, that segregation has dropped from 0.58 in 2000 to 0.45 in 2015. It is also worth mentioning that the decrease in segregation stopped in 2009 and remained almost completely stable until 2015, despite the increase in immigrant numbers throughout these years.
GRAPH II. Evolution of the percentage of foreign students in Compulsory Secondary Educa- tion in Spain and school segregation depending on nationality. Gorard index for first-generation immigrants
Source: Elaborated by the authors by PISA 2000, 2003, 2006, 2009, 2012 and 2015.
Revista de Educación, 381. July-September 2018, pp. 61-87 77 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Although the Gorard index is the most suitable for studying evolution since it is the only one that is not affected by the minority group size (Gorard, 2009; Murillo, 2016) we can see that this behaviour is supported in certain aspects with the calculations of the other indexes. The behaviour of the three indexes that measure the dimension of uniformity is very similar (with only minimal differences), whilst the results of the Isolation index, which measures exposure, is drastically different. This can be explained if we look at its nature: Isolation index estimates the probability of a foreign student coming across another student from the same country in his/her school. In this way, it is an index that is strongly influenced by the relative size of the minority group…the more foreigners there are the greater the probability that they will come across each other (table VII). The conclusion that we can draw from this is that the possible impact of the crisis has been reduced for school segregation due to national origin. It is true that the tendency to drop ceased from 2000 onwards, but its impact has not been as dramatic as it has for socioeconomic segregation.
TABLE VII. Evolution of school segregation depending on national origin in Spain for first-genera- tion immigrants
2000 2003 2006 2009 2012 2015 Gorard index 0.5775 0.5888 0.5002 0.4579 0.4580 0.4529 Dissimilarity index 0.5926 0.6060 0.5328 0.5000 0.5000 0.4980 Square Root index 0.4127 0.4152 0.3313 0.2697 0.2771 0.2535 Isolation index 0.1723 0.1169 0.2241 0.2102 0.2271 0.2296
Source: Elaborated by the authors by PISA 2000, 2003, 2006, 2009, 2012 and 2015.
An analysis of the evolution of school segregation depending on nationality in each region again offers such a rich perspective that it is difficult to describe it in such a small space (table VIII). Overall, two ideas are especially noteworthy. Firstly, we must again make mention of the wide range of variability between the segregation in the different regions, which cannot be fully explained by the different amounts of immigrant students. In this way,
78 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain although there is a high correlation between school segregation and the percentage of immigrants (rho = -0.629, sig. = 0.007), we can see that some autonomous communities have a far higher rate of segregation than expected (graph III).
GRAPH III. Relationship between the percentage of immigrant students and the school segrega- tion by national origin, as designed by Gorard index
Source: Elaborated by the authors by PISA 2015.
The evolutionary tendency of school segregation depending on nationality in each region varies dramatically. Therefore, although in all of them (and we must tread carefully as we do not have the full series in all cases) there is a tendency for segregation to drop, in five autonomous communities there is an increase between 2012 and 2015, in nine there
Revista de Educación, 381. July-September 2018, pp. 61-87 79 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
is a reduction, and there is not enough information to draw conclusions in the three remaining ones. The five autonomous communities in which segregation has increased in recent years, in order of the size of growth, are Extremadura, Navarre, Cantabria, Region of Murcia and the Balearic Islands. The Regions in which the segregation is lowest, on the other hand, can be listed in the following order: Castile and León, Community of Madrid, Andalusia, Aragon and the Basque Country. One explanation for these variations could be the percentage of immigrant students, so that an increase in the percentage of immigrant students is linked to a drop in Gorard’s index for segregation (rho = 0.613, sig. = 0.020).
TABLE VIII. Evolution of school segregation by national origin in Spain. Gorard index for first- generation immigrants in each autonomous communities
2000 2003 2006 2009 2012 2015 Andalusia 0.6791 0.5333 0.5224 0.4488 Aragon 0.4466 0.3070 0.3452 0.3019 Asturias 0.5907 0.4585 0.4079 0.3785 Balearic Islands 0.3901 0.3332 0.3432 Canary Islands 0.5045 0.4931 Cantabria 0.5118 0.3923 0.3660 0.3772 Castile and León 0.7049 0.6364 0.4365 0.4914 0.3721 Cast.-La Mancha 0.3439 Catalonia 0.5312 0.3721 0.3981 0.3975 0.3966 Extremadura 0.4987 0.6299 Galicia 0.7716 0.4005 0.4604 0.4451 La Rioja 0.3818 0.3465 0.2612 0.2562 C. of Madrid 0.3329 0.4004 0.3081 Reg. of Murcia 0.3623 0.2974 0.3076 Navarre 0.4415 0.4215 0.3243 0.3985 Basque Country 0.7892 0.7486 0.6075 0.5757 0.5375 Valencian C. 0.3759 Spain 0.5775 0.5888 0.5002 0.4579 0.4580 0.4529
Source: Elaborated by the authors by PISA 2000, 2003, 2006, 2009, 2012 and 2015.
80 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Discussion and Conclusions
This research has clearly shown that the work carried out to reduce school segregation in Spain over the last 15 years has been strongly interrupted by social and economic circumstances on a national scale in recent years. If we look at socioeconomic segregation and the segregation by national origin, the data continually shows that there is a clear tendency in recent years –at least in Spain as a whole– for school segregation to drop progressively from 2000 to 2012 and then increase suddenly in the last three years. The most probable explanation for this is the repercussions felt both in society and in the education system of the strong economic recession that began in 2007, the impact of which we will feel for some time. The fact that PISA analysed students of 15 years of age –that is, that they accessed the school at least 3-4 years ago–, may well explain a delay between the impact of the crisis on families and school segregation. This impact seems to be greater on the segregation of students from families of a greater economic, social and cultural status, lower amongst students from families of a higher economic, social and cultural status and has seen a sudden rupture in the tendency to decrease in segregation on account of nationality. That is to say, the information seems to indicate that students from families with lower levels of income have left those schools in which there are most students from families with higher levels of income, and vice versa. This serves as to enforce the notion of schools for the rich and schools for the poor, and has led Spain to become the sixth country with the higher segregation in the European Union (Murillo and Martínez-Garrido, 2018). It is risky to establish cause-effect relationships without considering the matter first. Indeed, from the fact that different tendencies can be seen in different regions we can deduce that other factors may be at play. In the Community of Madrid, for example, the area of the country with the greatest levels of socioeconomic school segregation, policies such as the introduction of a single district (distrito único), bilingual schools, the growth of private schools and the publication (more or less concealed) of school rankings as part of a clear tendency towards a quasi- market schooling, may well explain the increase in segregation. However, although we have no studies on the impact of the crisis on school segregation yet, similar studies on socioeconomic residential segregation have found that this has increased significantly in the years following the
Revista de Educación, 381. July-September 2018, pp. 61-87 81 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
crisis, both in other countries (e.g. Andersson and Hedman, 2016; Zwiers Bolt, Van Ham and Van Kempen, 2016), and in Spain (Bayona-i-Carrasco, Gil-Alonso, Rubiales-Pérez and Pujadas-Rúbies, 2017; Leal and Sorando, 2016). Given the close relationship between both phenomena (Lareau and Goyette, 2014), it all makes us believe that the crisis has a key part to play in this obvious growth. In any case, more research is needed to look into this hypothesis further. School segregation directly threatens equal opportunities for all and, in this way, leads to a society that is less united and less equal. The fact that there are many factors at play behind segregation (Gorard y Fitz, 2000) means that it is particularly difficult to fight against it. Therefore, although there are factors that are difficult to change in the short term (such as the residential segregation we have already mentioned), educational policies exist that can increase or reduce school segregation. The cuts to public education in terms of reducing the numbers of teachers, specialists and their working conditions, as well as the quality of their installations, increasing student numbers and the impact that this has, deliberate or not, on the social image of public schools to the detriment of private schools. For example, it can cause the election of moving into some schools and, in turn, causes segregation. If the financial crisis really has come to an end, we must make political decisions that compensate for this and seek for a return to the tendency for segregation to drop. We cannot forget to mention that the use of PISA databases for a study on school segregation has its limitations. Before we go through these, however, it is important to note not only that PISA is the only source of data available that allows us to make a evolutionary study (since it is carried out every three years using the same method) like this one, but also that gives us access to an insight into school segregation that is relatively reliable. We must not forget that information from 94,464 students at 3,235 schools is analysed over the course of 15 years in by PISA. One limitation of the programme, without doubt, is that by only using a few students per school, segregation between classrooms may be concealed, as may other types of segregation within each school. This research is merely a first look into a worrying reality that has hardly been studied in Spain, but it is only that. Future research is needed to help identify which educational policies favour school segregation, studies that look into the role of private education in school segregation and that look into the impact of segregation on the academic success
82 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain of students. These studies should look at the development of values of coexistence and contribute towards a greater understanding of school’s selection criteria by families as well as the impact of the crisis, amongst other factors. A segregated education system inevitably leads to an unequal and unjust society. The financial crisis has led to Spain becoming one of the most unequal countries in Europe. If the crisis really has come to an end, we must take urgent measures for the situation to revert back to one of school segregation in decline. If we do not, the dream towards a fairer and inclusive society will be further and further from our reach.
References
Alegre, M. A., Benito, R., & González-Baetbó, I. (2008). Procesos de segregación y polarización escolar: La incidencia de las políticas de zonificación escolar [Segregation and school polarization processes: The incidence of school zoning policies]. Profesorado. Revista de Currículum y Formación del Profesorado, 12(2), 1-26. Andersson, R., & Hedman, L. (2016). Economic decline and residential segregation: A Swedish study with focus on Malmö. Urban Geography, 37(5), 748-768. https://doi.org/10.1080/02723638.2015.1133993 Ayllón, S. (2015). Infancia, pobreza y crisis económica [Childhood, poverty and economic crisis]. Barcelona: Obra Social “La Caixa” Bayona-i-Carrasco, J., Gil-Alonso, F., Rubiales-Pérez, M., & Pujadas-Rúbies, I. (2017). New spatial mobility patterns in large Spanish cities: From the economic boom to the great recession. Applied Spatial Analysis and Policy, 10, 1-26. https://doi.org/10.1007/s12061-017-9222-x Benito, R., & González-Baetbó, I. (2007). Processos de segregació escolar a Catalunya [Processes of school segregation in Catalonia]. Barcelona: Mediterrània. Bonal, X. (2012a). Municipis contra la segregació escolar. Sis experìències de política educativa local [Municipalities against school segregation. Six experiences of local educational policy]. Barcelona: Fundació Jaume Bofil.
Revista de Educación, 381. July-September 2018, pp. 61-87 83 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Bonal, X. (2012b). Education policy and school segregation of migrant students in Catalonia: The politics of non-decision-making. Journal of Education Policy, 27(3), 401-421. https://doi.org/10.1080/02680939.2 011.645168 Cebolla-Boado, H., & Garrido Medina, L. (2011). The impact of immigrant concentration in Spanish schools: School, class, and composition effects. European Sociological Review, 27(5), 606-623. https://doi. org/10.1093/esr/jcq024 Coleman, J., Hoffer, T., & Kilgore, S. (1982). High school achievement: Public, catholic and private schools compared. New York, NY: Basic Books. Duncan, O., & Duncan, B. (1955). A methodological analysis of segregation indexes. American Sociological Review, 20, 210-217. https://doi. org/10.2307/2088328 Ferrer, F. (Dir.). (2008). Les desigualtats educatives a Catalunya [Educational inequalities in Catalonia]. Barcelona: Fundació Jaume Bofill. Ferrer, F., Castejón, A., Castel, J. L., & Zancajo, A. (2011). PISA 2009: Avaluació de les desigualtats educatives a Catalunya [PISA 2009: Evaluation of educational inequalities in Catalonia]. Barcelona: Fundació Jaume Bofill. García-Gómez, S., & Cabanillas, M. C. (2017). Incidencia de la crisis económica en el alumnado de educación primaria. La perspectiva del profesorado [Incidence of the economic crisis in primary education students. The perspective of the teaching staff]. Revista Complutense de Educación, 28(1), 185-201. https://doi.org/10.5209/rev_RCED.2017. v28.n1.49075 Gorard, S. (2009). Does the index of segregation matter? The composition of secondary schools in England since 1996. British Educational Research Journal, 35(4), 639-652. https://doi.org/10.1080/01411920802642389 Gorard, S., & Fitz, J. (2000). Investigating the determinants of segregation between schools. Research Papers in Education, 15(2), 115-132. https://doi.org/10.1080/026715200402452 Gorard, S., & Taylor, C. (2002). What is segregation? A comparison of measures in terms of ‘strong’ and ‘weak’ compositional invariance. Sociology, 36(4), 875-895. https://doi.org/10.1177/003803850203600405 Gorard, S., Hordosy, R., & See, B. H. (2013). Narrowing down the determinants of between-school segregation: An analysis of the intake
84 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
to all schools in England, 1989-2011. Journal of School Choice, 7(2), 182-195. https://doi.org/10.1080/15582159.2013.791182 Hutchens, R. (2004). One measure of segregation. International Economic Review, 45(2), 555-578. https://doi.org/10.1111/j.1468- 2354.2004.00136.x Lareau, A., & Goyette, K. (2014). Choosing homes, choosing schools. New York, NY: Russell Sage Foundation. Leal, J., & Sorando, D. (2016). Economic crisis, social change and segregation processes in Madrid. In T. Tammaru, S. Marcińczak, M. Van Ham & S. Musterd (Eds.), Socio-economic segregation in European capital cities: East meets west. London: Routledge. Lieberson, S. (1981). An asymmetrical approach to segregation. In C. Peach (Ed.), Ethnic segregation in cities (pp. 61-83). London: Croom- Helm. Mancebón-Torrubia, M. J., & Pérez-Ximénez, D. (2009). Segregación escolar en el sistema educativo español. Un análisis a partir de PISA 2006 [School segregation in the Spanish educational system. An analysis from PISA 2006]. Investigaciones de Economía de la Educación, 4, 63-77. Mancebón-Torrubia, M. J., & Pérez-Ximénez, D. (2010). Una valoración del grado de segregación socioeconómica existente en el sistema educativo español. Un análisis por comunidades autónomas a partir de PISA 2006 [An assessment of the degree of socioeconomic segregation existing in the Spanish educational system. An analysis by autonomous communities from PISA 2006]. Regional and Sectoral Economic Studies, 10(3), 129-148. Mas Ivars, M. Pérez García, F., & Uriel, E. (Dirs.). (2015). Capital público en España: Evolución y distribución territorial (1900-2012) [Public capital in Spain: Evolution and territorial distribution (1900-2012)]. Bilbao: Fundación BBVA. Murillo, F. J. (2016). Midiendo la segregación escolar en América Latina. Un análisis metodológico utilizando el TERCE [Measuring school segregation in Latin America. A methodological analysis using TERCE]. REICE. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 14(4), 33-60. https://doi.org/10.15366/reice2016.14.4.002 Murillo, F. J., & Martínez-Garrido, C. (2018). Magnitud de la Segregación escolar por nivel socioeconómico en España y sus Comunidades Autónomas y comparación con los países de la Unión Europea
Revista de Educación, 381. July-September 2018, pp. 61-87 85 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
[Magnitude of the school segregaton by socioeconomic level in Spain and its autonomous communities and comparison with the countries of the European Union]. RASE: Revista de la Asociación de Sociología de la Educación, 11(1), 37-58. https://doi.org/10.7203/RASE.11.1.10129 Murillo, F. J., Martínez-Garrido, C., & Belavi, G. (2017). Segregación escolar por origen en España [School segregation by origin in Spain]. OBETS, 12(2), 395-423. https://doi.org/10.14198/OBETS2017.12.2.04 OCDE. (2016). Programme of international student assessment (PISA 2015). Paris: OCDE. Orfield, G., Ee, J., Frankenberg, E., & Siegel-Hawley, G. (2016). Brown at 62: School segregation by race, poverty and state. Los Angeles, CA: Civil Rights Project, UCLA Owens, A., Reardon, S. F., & Jencks, C. (2016). Income segregation between schools and school districts. American Educational Research Journal, 53(4), 1159-1197. https://doi.org/10.3102/0002831216652722 Reardon, S. F., & Owens, A. (2014). 60 Years after Brown: Trends and consequences of school segregation. Annual Review of Sociology, 40, 199-218. https://doi.org/10.1146/annurev-soc-071913-043152 Sánchez Hugalde, A. (2007). La segregació escolar dels immigrants a Catalunya [Immigration segregation in Catalonia]. Quaderns D’avaluació, 13, 55-76. Síndic. (2008). La segregación escolar en Cataluña [School segregation in Catalonia]. Barcelona: Síndic. Stephan, W. (Ed.). (2013). School desegregation: Past, present, and future. New York, NY: Springer. Stroub, K. J., & Richards, M. P. (2013). From resegregation to reintegration: Trends in the racial/ethnic segregation of metropolitan public schools, 1993-2009. American Educational Research Journal, 50(3), 497-531. https://doi.org/10.3102/0002831213478462 Valiente, O. (2008). ¿A qué juega la concertada? La segregación escolar del alumnado inmigrante en Cataluña (2001-06) [What is the role of private dependent schools? The school segregation of immigrants in Catalonia (2001-06)]. Profesorado. Revista de Currículum y Formación del Profesorado, 12(2), 1-23. Van-Zanten, A. (2007). School differentiation and segregation in the Parisian periphery: An analysis of urban schools’ logics of action and their effects. In W. Pink & G. Noblit (Eds.), International handbook of urban education (pp. 431-445). Dordrecht: Springer.
86 Revista de Educación, 381. July-September 2018, pp. 61-87 Received: 04-09-2017 Accepted: 23-02-2018 Murillo, F. J., Martínez-Garrido, C. Impact of the economic crisis on school segregation in Spain
Zwiers, M., Bolt, G., Van Ham, M., & Van Kempen, R. (2016). The global financial crisis and neighborhood decline. Urban Geography, 37(5), 664-684. https://doi.org/10.1080/02723638.2015.1101251
Contact address: F. Javier Murillo. Universidad Autónoma de Madrid, Facultad de Formación de Profesorado y Educación. Departamento de Didáctica y Teoría de la Educación. Universidad Autónoma de Madrid. C/ Tomas y Valente 3. Madrid E-28049, España. E-mail: [email protected]
Revista de Educación, 381. July-September 2018, pp. 61-87 87 Received: 04-09-2017 Accepted: 23-02-2018
Analysis of the Learning Styles measurement tools1
Análisis de las herramientas de medición de los Estilos de Aprendizaje
DOI: 10.4438/1988-592X-RE-2017-381-382
María Luz Diago Egaña María José Cuetos Revuelta Patricia González González Universidad Internacional de la Rioja
Abstract The first time the “Learning Style” construct (Gibson, 1969) appears, it does so with a meaning that denotes how a person prefers to be presented with information to carry out the learning. This definition has evolved over the years, while numerous tools were developed to detect each of its dimensions. This study aims to develop an updated list of tools available for the detection of different learning styles, analyse its metric characteristics and to weigh the questionnaires according to their use, as well as to analyse the relationship between the measurement tools of the learning styles. A comprehensive bibliographic search is carried out by using different internet search engines such as Google Scholar, Web of Science (WOS), Dialnet, Education Resources Information Center (ERIC), ScienceDirect and a bibliometric analysis was conducted in the period from 21 May 2015 to 30 April 2017. We present an updated list of the tools available to detect learning styles, based on the four minimum criteria postulated by Coffield, Moseley, Hall and Ecclestone (2004). We analyse the metric characteristics of the questionnaires collected by them, together with others found in our research, and we consider whether or not these questionnaires are validated.
(1) This work has been supported by Universidad Internacional de la Rioja (UNIR). It is part of the research project entitled “Learning Styles applied to Experimental Sciences” granted by UNIR after internal call for research projects in July 2016.
Revista de Educación, 381. July-September 2018, pp. 89-125 89 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
While the CSI questionnaire (Allison and Hayes, 1996) is presented as the most robust, if we stick to questionnaires that would fit with the original semantics of the term “Learning Styles” (Gibson, 1969), the inventories VAK/VARK (Google Scholar in any language), CHAEA (Google Scholar in Spanish) and ILS (Felder and Soloman, 1997) (Web of Science and ScienceDirect) would be the most widely used. It is concluded that the percentage weight of the questionnaires, based on their presence in literature, varies according to the search language and the search engine used.
Keywords: Learning styles, perception channels, learning strategies, learning preferences, learning styles, cognitive styles, neuroscience, questionnaire.
Resumen La primera vez que aparece el constructo “Estilo de Aprendizaje” (Gibson, 1969), lo hace con un significado que denota el modo en que una persona prefiere que le presenten la información para llevar a cabo el aprendizaje. Esta definición ha ido evolucionando a lo largo de los años, al mismo tiempo que se desarrollaban numerosas herramientas para detectar cada una de sus dimensiones. En este estudio se pretende elaborar un listado actualizado de herramientas disponibles para la detección de los distintos estilos de aprendizaje; analizar sus características métricas, la relación entre ellos y ponderar los cuestionarios en función de su utilización. Para ello se realiza una búsqueda bibliográfica exhaustiva en distintos buscadores como Google académico, Web of Science (WOS), Dialnet Education Resources Information Center (ERIC), ScienceDirect y un análisis bibliométrico en el periodo comprendido entre el 21 de mayo de 2015 al 30 de abril de 2017. Mostramos un listado actualizado de las herramientas disponibles para detectar los estilos de aprendizaje, basándonos en los cuatro criterios mínimos postulados por Coffield, Moseley, Hall y Ecclestone (2004). Se analizan las características métricas de los cuestionarios recogidos por ellos juntamente con otros encontrados en nuestra investigación, considerando también si dichos cuestionarios están o no validados. Mientras que, el cuestionario CSI (Allison y Hayes, 1996) se presenta como el más robusto, si nos ceñimos a los cuestionarios que encajarían con la semántica original del término “Estilos de Aprendizaje” (Gibson, 1969) los que se revelan con más presencia en diferentes motores de búsqueda serían en VAK/VARK (Google Scholar en cualquier idioma), CHAEA (Google Scholar en español) e ILS (Felder y Soloman, 1997) (Web of Science y ScienceDirect). Se concluye que la ponderación porcentual de los cuestionarios basándonos en su aparición según diferentes buscadores, varía en función de la lengua de búsqueda y el buscador utilizado.
Palabras clave: Estilos de aprendizaje, canales de percepción, estrategias de aprendizaje, preferencias de aprendizaje, estilos cognitivos, neurociencia, cuestionario.
90 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Introduction
Learning Styles (LSs) have significant influence within the field of education and they have been studied for many years, focusing on pre-school to university levels. One of the main aims of studying and determining LSs was, for many years, to generally improve both the immediate and the long-term results of the teaching-learning process. In fact, over the last 40 to 50 years, a thriving industry, dedicated to publishing tests and guides for teachers, has been created around the LS concept. Furthermore, many organisations in the industry offer professional development workshops for teachers and educators based on said concept (Pashler, McDaniel, Rohrer and Bjork, 2008). In all of these actions, the fundamental idea that lies behind this construct is that each one of us has a specific learning style (sometimes called a “preference”) and that we learn better when we are presented with information in that style. There is an extensive, and somewhat controversial, amount of literature on LSs, starting with the very definition of the construct. In general, defenders of the theory of learning styles uphold that the best education for students requires an assessment to be undertaken on the learning styles of the individuals and, consequently, the form of education to be adapted to them. Having knowledge of the learning styles used by students allows teachers to discover what methodology to use and how to carry it out in the most effective way. Some authors suggest that it has been sufficiently proven that students learn more effectively when they are taught using their own predominant LS (Alonso, Gallego and Honey, 1999). However, according to Alonso et al. (1999) consideration has to be given not only to the LS of students, but also to the Teaching Style of teachers. In this respect, some studies point to the need to analyse the LSs of students and to relate them to the teaching styles used by teachers (Coloma, Manrique, Revilla and Tafur, 2008; Pupo, 2012; González- Peiteado, 2013; Aiello, García, and Jaramillo, 2015), as it may facilitate the possibility of the students’ academic success (Saarikoski, Salojärvi, Del Corso and Ovcin, 2001). However, Pashler et al. (2008) found, despite the fact that there is a vast amount of literature on learning styles, that there is practically no evidence that supports the idea that if education is provided in a way that coincides with the preference of students, their academic performance improves. Those authors claim that the design
Revista de Educación, 381. July-September 2018, pp. 89-125 91 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
of most of the research conducted is weak and that some research studies with an effective experimental design even obtained results that categorically contradicted the popular assumptions on learning styles. In fact, some of the weaknesses corresponded to the design of the very tools used to measure the learning styles. Over the years, numerous attempts have been made to determine the LSs of students through the designing of questionnaires. Several measurement tools to detect all of the dimensions of the LSs have progressed as the definition of the construct has evolved. Therefore, we find questionnaires that measure different parameters, such as personality (MSP of Apter, Mallows and Williams, 1998; MBTI (Myers and McCaulley, 1985), interaction within the group (SLSQ of Grasha, 1996), cognitive level (LSI of Kolb, 1984), emotional level (CHAEA of Alonso, Gallego and Honey, 1994), method of processing information (left and right hemisphere, the HBDI of Herrmann, 1982), environment (Dunn, Dunn and Price, 1989), information perception channel (VARK of Fleming and Mills 1992), response time (MFFT of Kagan, 1966), learning context (questionnaire of Grasha, 1996), the basic preferences in relation to goals, attitudes and feelings (LIFO questionnaire of Atkins, cited in García, Santizo and Alonso, 2009), among others. In view of the foregoing, it is apparent that there is a significant number of tools available for determining different learning styles, based on multiple aspects and examined at different educational stages, predominantly relating to studies conducted on higher education rather than on primary and secondary education. Furthermore, these models have not been without their criticisms, particularly due to the lack of reliability and validity, according to Curry (1990) and Coffield et al. (2004), of the measurements undertaken.
Aims of the study
The main aim of this study is to draw up a list of the tools currently available for detecting the different Learning Styles, as well as to analyse their metric characteristics, and, secondly, to analyse which questionnaires are most used by different authors to measure LSs and the connection that exists between the measurement tools.
92 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Method
The methodology used in this study is based on an extensive bibliographic review using the following databases: Scholar Google, Web of Science (WOS), Dialnet, ERIC and ScienceDirect.
Process
This study commences with the selection of the 13 questionnaires that Coffield et al. established as the main ones in 2004. A thorough bibliographic review was undertaken from 21 May 2015 to 30 April 2017 using each keyword that appears in Table I (Annex) as search criteria, using them either individually or combinations of them. After reading the immense volume of articles found, those that directed us towards measurement instruments were filtered. The selection criteria of said instruments included those that measured learning styles and that were either not originally considered by Coffield et al (2004) or, if they were, did not feature as the main ones, or that were published after said study. Furthermore, it was taken into account that the construct of those instruments was consistent with the LS definition of Gibson (1969). Once the instruments had been chosen in such manner, their names and acronyms (Table I) were used to analyse each instrument. The purpose of the analysis was to establish the construct and the dimensions, to verify whether or not the instrument had been validated and, if so, who had validated it, and the psychometric characteristics. The bibliographic analysis on the questionnaires most used in recent years was undertaken using Google Scholar, Web of Science and ScienceDirect search engines, in both Spanish and English, while taking into consideration the following criteria: • Full name of the instrument in question • Initials of the instrument + surname of author(s) The average of these searches is the quantitative datum obtained, which is set out in the different graphs in the results section.
Revista de Educación, 381. July-September 2018, pp. 89-125 93 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Results and Discussion
Learning Style detection tools
Many tools for detecting Learning Styles have been designed since the concept first started to be discussed in the 1950s. In 2004, Coffield et al conducted an analysis on 71 questionnaires that were around at the time to measure LSs. They categorised 13 as the main questionnaires and concluded that, of those 13, only the Cognitive Style Index (CSI) (Allinson and Hayes, 1996) met the four minimum criteria that are essential for a questionnaire to be used in successive research (internal consistency, test- retest reliability, construct validity and predictive validity). Conversely, the questionnaires of Riding (1991) (CSA: Cognitive Styles Analysis) and Sternberg (1999) (TSI: Thinking Styles Inventory), failed to meet any of the essential minimum criteria and, therefore, they were not considered in this study. Table II (Annex) contains information on the CSI (Coffield et al. 2004). The study conducted by Coffield et al. (2004) also suggests that two questionnaires, the MSP (Motivational Style Profile) of Apter, Mallows and Williams (1998) and the ILS (Inventory of Learning Styles) of Vermunt (1996), meet 3 out of the 4 minimum criteria required in a questionnaire. The questionnaire of Apter et al. (1998) meets the criteria of internal consistency, test-retest reliability and predictive validity, while the questionnaire of Vermunt (1996) meets the criteria of internal consistency, test-retest reliability and construct validity (Table III, annex). Similarly, the study of Coffield et al. (2004) suggests that 3 further questionnaires, those of ASSIST (Approaches and Study Skills Inventory for Students) of Entwistle (1988), HDBI (Herrmann’s Brain Dominance Instrument) of Herrmann (1989) and the MBTI (Myers-Briggs Type Indicator) (Myer and McCaulley, 1985, 1998), meet 2 of the 4 minimum criteria needed. Finally, Coffield et al. (2004) suggests that 4 further questionnaires meet 1 out of the 4 minimum criteria needed, as set out in Table IV (Annex). Additionally, the LSP questionnaire (Jackson’s Learning Styles Profiler) of Jackson (2002) (Table IV), which was not initially validated by Coffield et al. (2004) due to a lack of data, proved a posteriori to be valid in 1 out of the 4 criteria (Table IV).
94 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Therefore, as can be seen in Table III, of the tools validated by Coffield et al. (2004) the CSI questionnaire measures information processing preferences and the HDBI, LSQ and LSI questionnaires measure learning preferences, while the VAK/VARK and GSD questionnaires measure modalities in the perception of information and specific channels for processing information, respectively. These tools are the ones that relate to our construct. Besides the questionnaires analysed in Coffield et al. (2004), other questionnaires used for measuring LSs have been sought. The information obtained in that regard can be seen in Table V (Annex), where the metric characteristics of the questionnaires are set out. As can be observed in said Table V, the only questionnaires used to measure learning preferences are Barsch’s Test, the ILS (Felder- Soloman), the LSS and the CESEA (the Spanish acronym for Learning Styles according to the questionnaire by Escanero and Soria). We will now analyse which questionnaires measuring learning preferences, perception channels and cognitive styles have continued to be used in recent years. The most recent studies that have used those instruments in the methodology of field research are set out in Table VI (Annex). In addition to Table VI and with the aim of studying the impact of using the 11 questionnaires, which, according to Coffield et al. (2004), present 1-4 minimum criteria to be validated, a search was undertaken through Google Scholar using the names of different questionnaires and delimiting the search from the publication of the corresponding study, in other words, the period from 2004 to 2016, and on the entire internet. The results of said search are set out in Graph I. It can be observed in Graph I that the CSI, which Coffield et al. (2004) suggests met all four minimum criteria that are essential for a questionnaire to be used in subsequent research, only appeared in 2% of the studies found. Conversely, and with a significant difference, the MBTI questionnaire (measuring characteristics of personality and cognitive structure) was referred to in 44% of the studies found in Google Scholar during the period in question. If we exclusively focus on the questionnaires related to our construct, then we can see in this graph that the LSI and the VAK/VARK were reflected in a greater number of the studies found, specifically, 15% and 17%, respectively. It is worth mentioning that despite the fact the
Revista de Educación, 381. July-September 2018, pp. 89-125 95 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
LSI questionnaire measures LS preferences, it only met 1 out of the 4 criteria established by Coffield et al. (2004) to be validated, and it was not validated by authors such as Reynolds (2003) and Shum (2003). Likewise, the VAK/VARK questionnaire measures modalities in the perception of information and, as with the previous one, it only met 1 criterion to be validated (Coffield et al. 2004), but, despite being validated a posteriori by Leite, Svinicki and Shi (2010), they advise the exercising of caution in its research use regarding the drawing up of items and the scoring of scale algorithms. Furthermore, according to Fleming (2012), the VAK/ VARK does not technically refer to LSs, as the questionnaire only provides information on the preferred communication modalities.
GRAPH I. Results of the search for different questionnaires in any language (n=10167 results obtained).
AVERAGE SEARCH RESULTS FOR11COFFIELD QUESTIONNAIRESIN ANY LANGUAGE (Google Scholar) GSD LSP CSI HBDI 2% 1% 2% 1% LSQ 2% LSI VAK/VARK 15% 17%
MSP 0% ILS 1%
ASSIST 14%
MBTI 44%
Source: produced by the author(s) of this study
96 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
The results obtained from the next search, undertaken in the same way, but this time only in Spanish, can be analysed in Graph II.
GRAPH II. Results of the search for different questionnaires only in Spanish. (n=930 results obtained).
AVERAGE SEARCH RESULTS FOR11COFFIELD QUESTIONNAIRESIN SPANISH (Google Scholar)
GSD LSP CSI 0% 0% 1% HBDI LSI 4% 16%
MBTI LSQ/CHAEA 13% 37%
ASSIST 3%
ILS 2% MSP 0%
VAK/VARK 24%
Source: produced by the author(s) of this study
It can be seen in Graph II that in Spanish, the CSI even appears in a lower percentage than in the any language search, while the presence of the CHAEA (the Honey-Alonso Learning Styles Questionnaire adapted into Spanish, Honey and Mumford (1982)) considerably increases, going from 2% to 37%, followed by the VAK/VARK with 24%. It is worth remembering that Coffield et al. (2004) only found, in both questionnaires, 1 minimum criterion required for them to be used.
Revista de Educación, 381. July-September 2018, pp. 89-125 97 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
The results obtained when we added the questionnaires reflected in Table V to those analysed by said authors are as follows. Thus, Graph III shows, for the same 2004-2016 period, the overview of results on the entire internet when we consider all of the questionnaires, those analysed by Coffield et al. (2004) and those studied in said table. In this search, the LSS was omitted because it was designed in 2014 and up to 2016 it had only been used by its authors.
GRAPH III. Results of the search for all questionnaires using the entire internet (n=12688 results obtained).
AVERAGE SEARCH RESULTS FOR ALL QUESTIONNAIRESIN ANY LANGUAGE (Google Scholar) HBDI CESEA CSI LSQ FS 1% 0% 2% 2% 8% BLSI CEPEA VAK/VARK 2% 1% 13%
SPQ 9% MSP 0% ILS LSP 1% 1% GSD 1%
ASSIST 11%
LSI 12%
MBTI 35%
Source: produced by the author(s) of this study
As seen in Graph III, the questionnaires most used are the MBTI, LSI, VAK/VARK and the ASSIST, while the SPQ and the ILS questionnaires
98 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
(Felder-Soloman, appearing as FS in the graph) appear in this analysis with a marginally significant presence of 9% and 8%, respectively. As regards the tools related to our construct, the LSI is not validated (Coffield et al. only found 1 criterion of the 4 required) and, as previously mentioned, caution is advised in using the VAK/VARK, even though it was validated in 2010 by Leite et al. The ILS questionnaire (Felder-Soloman), measuring learning preferences or tendencies, has recently been validated (Zywno, 2003; Felder and Spurlin, 2005; Hosford and Siders, 2010; Wang and Mendori, 2015). As such, it could be used as a tool for measuring learning preferences. To be more rigorous in this study and with the aim of further extending it, the searches conducted in Google Scholar were also undertaken using specialist search engines, namely, Web of Science (WOS) and ScienceDirect. The only difference compared with the previous study, is that no differentiation was made regarding language. The results obtained can be seen in Graph IV. Data relating to the CEPEA questionnaire do not feature in Graph IV as no record of it was found using these search engines. Also not shown, are data relating to the CESEA, as only two records of it were found in WOS and none in ScienceDirect. If we focus exclusively on the questionnaires related to our construct, those that appear with a higher presence are, in order, the LSI and the ILS (Felder-Soloman). As regards the LSI, even though it measures preferences of Experiential Learning Styles, it was not validated by Reynolds (2003) or Shum (2003), but it was by Coffield et al. (2004), who suggested that it only met 1 out of the 4 criteria. In fact, different versions of the LSI questionnaire have continued to be developed over the years. The most recent is the KLSI version 4 (2013), but it has only been validated by its authors. Therefore, the search in other databases –WOS and ScienceDirect– leads us once again to the ILS questionnaire (Felder-Soloman) as a candidate to be used for measuring LSs, due to the fact that it measures learning preferences or tendencies, it is one of the most used and that it has been recently externally validated. Similarly, although the CSI questionnaire appears to be one of the least used in recent years, it is the only one that meets the four criteria established by Coffield et al. (2004) and it has also been recently validated by Armstrong and Qi (2016).
Revista de Educación, 381. July-September 2018, pp. 89-125 99 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
GRAPH IV. Results of the search for all questionnaires using specialist search engines. (WOS n=837 results obtained; ScienceDirect n=1046 results obtained).
AVERAGE SEARCH RESULTS FOR ALL QUESTIONNAIRESIN SPECIALISTSEARCH ENGINES
40,0
35,0 MBTI SPQ 30,0
ge ta 25,0
cen
per 20,0 LSI
esult
r 15,0
ch ILS Sear 10,0 VAK/ VARK ILS 5,0 CSI HBDI LSQ ASSIST MSP GSD BLSI LSP 0,0 %%%%%%%%%%%%%% WOS 2,4 0,9 2,2 0,7 0,4 3,4 1,7 32,1 15,8 0,9 30,4 0,3 8,1 0,7 Science Direct 2,3 1,0 2,2 4,2 0,5 3,5 1,8 30,3 16,9 0,5 29,5 0,1 6,9 0,4
Source: produced by the author(s) of this study
Nevertheless, we also found critics for both questionnaires. Thus, regarding the ILS questionnaire (Felder-Soloman), Al-Azawei, Parslow and Lundqvist (2015) consider it, in general, valid for analysing LSs, but also point to the moderate robustness of the criteria analysed (reliability and validity of perception) or low internal consistency as regards some of the constructs. What is more, Çardak and Selvi (2016) show the need to conduct further studies that confirm the validity of the questionnaire, given that the results of the validation test differ according to the cultures the students are from, a fact previously highlighted by Joy and Kolb (2009). In terms of the CSI questionnaire, even though the results shown in the internal reliability tests and reliability test-retest were strong (Coffield et al. 2004), other researchers have criticised the structure of underlying factors of the initial questionnaire. While, for some, its nature is unifactorial (Allinson and Hayes, 1996; Armstrong and Qi, 2016), others suggest that
100 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools its nature is clearly multifactorial, where the two dimensions, intuition and analysis, have to be handled in a separate but correlated manner (Hodgkinson and Sadler-Smith, 2003; Backhaus and Liff, 2007). Backhaus and Liff, in their 2007 paper, analysing the nature of said structure, used this questionnaire in a sample of 222 American university students studying business, and differences were found in the results obtained compared with what Allinson and Hayes observed in their 1996 British study. One possible reason for those differences is attributed to the drawing up of the questionnaire items as, on reading it, the American students may interpret the contents differently given that they are used to a different version of English. Due to the results obtained, the authors concluded the possibility that this instrument may not have been able precisely measure how that specific sample processed information, considering it necessary to revise the semantics of the questionnaire to eliminate any disparity in the use of the language. This conclusion also coincides with that previously stated by Hodgkinson and Sadler-Smith (2003), who established that the scoring process and issues in drawing up the CSI questionnaire gave rise to unsatisfactory results and, therefore, need to be revised. Moreover, in order for the results obtained, on using this questionnaire, to be as representative as possible, the sample size to study must be extensive and as heterogeneous as possible (Backhaus and Liff, 2007).
Connection between some of the tools used for measuring LSs
The connection between some of the tools that have been used over the years for detecting LSs is shown below: The ILS (Felder-Soloman) has a version adapted into Spanish by Martínez-Fernández, García-Ravidá, González-Velázquez, Gutiérrez- Braojos, Poggioli, Ramírez-Otalvaro and Tellería (2009). This questionnaire aims to identify the strategies, motivations and attitudes in terms of studying or learning tasks. The questionnaire comprises two parts. The first focuses on identifying, from a list of 55 activities, those that students undertake in the context of their studies. The second part (24 items) focuses on analysing motives, objectives and attitudes that students have as regards their studies.
Revista de Educación, 381. July-September 2018, pp. 89-125 101 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
The LSQ was adapted into Spanish, using the title CHAEA (Honey- Alonso Learning Styles Questionnaire). Those authors divide the LSs into four categories: active, reflective, theoretic and pragmatic. Baus (2007) further extended and supplemented those definitions, establishing that, in the active style, the individual improvises, takes risks, discovers and is spontaneous, and, therefore, he or she is a dynamic student in class. The reflective style refers to receptive, analytical and observant people. The pragmatic style is characterised by students who are quick, determined, plan-makers, specific, and who have well-defined and solid objectives. Lastly, people who have a theoretic style are characterised as being disciplined, systematic, ordered, mechanised, reasoners, thinkers, perfectionists and seekers of theoretic models. Dunn, Dunn and Price (1975), originally established three modalities, called perception channels: visual, auditory and kinaesthetic (VAK). Subsequently, Fleming and Mills (1992) increased the modalities by adding “Reading/writing” (VARK). On reading the corresponding literature, it is not easy to determine whether or not those channels are considered learning styles (Gamboa-Mora, Briceño-Martínez and Camacho-González, 2015). According to Fleming and Baume (2006), and only from a theoretical perspective, those modalities could only be considered to be one part of a bigger whole for them to be included in the Learning Styles. Furthermore, Fleming (2012), is of the opinion that technically VARK is not a Learning Style, but rather merely refers to how people learn and focuses on the different modalities in which people would prefer to learn. As such, said questionnaire would only provide information on preferred communication modalities and its results would only be indicative, rather than diagnostical. The fact that an individual shows a high preference for one of the channels, does not mean that the other modalities are non-existent in that individual. The connection between the approaches of the CHAEA and the VAK is determined according to the content sought to be taught in class (Quiñones, 2004). According to this author, the teaching strategies determined must take into consideration the different perception channels (visual, auditory and kinaesthetic), as they act as precursors in the development of different learning styles (active, reflective, theoretic and pragmatic). The approach of the Neuro-linguistic Programming (NLP), as well as of Dunn, Dunn and Price (1985), establishes that people perceive
102 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools the world from the three aforementioned perception channels: visual, auditory and kinaesthetic. However, it adds that individuals largely tend to develop one more than the other, giving rise to a leading perception channel. Thus, it may be concluded that the theory in the VAK, concurs with the developments of Neuro-linguistic Programming (NLP), whereas the CHAEA does not (Gamboa et al. 2015). Furthermore, Escanero-Marcén, Soria, Guerra-Sánchez and Silva (2016) conclude that the CHAEA and the Felder-Silvermann questionnaire are not completely effective, and, as such, have drawn up a new questionnaire based, like previous ones, on Kolb’s experiential learning and on the cognitive styles of Allison and Hayes (1996). The new questionnaire is called: CESEA (Learning Styles according to the Questionnaire by Escanero and Soria). When preparing and selecting the items comprising the CESEA, the authors took into account the cognitive model of the Cognitive Style Index (CSI). According to Escanero-Marcén et al. (2016) the equivalency between the CHAEA styles and the poles of the CESEA, are as follows: The four styles explored in the CHAEA, and that are represented on the four axes of the diagram, are the active, reflective, theoretic and pragmatic styles, which correspond with the dynamic, reflective, theoretic and operative poles, respectively, of the CESEA. On the other hand, the styles in the CESEA are quadrantes and they have the properties of the two poles that delimit them.
Conclusions
The conclusions of this study are as follows: Of the 15 questionnaires analysed in this paper, only the CSI, HDBI, LSQ, LSI, VAK/VARK, GSD, Barsch’s Test, ILS (Felder-Soloman), LSS and the CESEA questionnaires measure learning preferences, perception channels and cognitive styles. As such, they are the ones that we consider relate to our construct. Of those questionnaires, only the CSI meets the four criteria (internal consistency, test-retest reliability, construct validity and predictive validity) according to Coffield et al. (2004). On analysing the presence/relevance on the internet of those questionnaires, we found that there are some differences, as regards the relevance of each paper, when we search for studies by language (on the entire internet or only in Spanish) and when we search using search
Revista de Educación, 381. July-September 2018, pp. 89-125 103 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
engines that are more or less specialised (Google Scholar or WOS and ScienceDirect). Regardless of the language and the search engine used, the presence/ relevance on the internet of the CSI questionnaire is only between 1-2.5%, despite being validated by different authors. Nevertheless, others criticise its structure and the scoring process, and point to issues regarding the drawing up of the items. In the other questionnaires analysed by Coffield et al. (2004) that are directly related to the LSs, we found the VAK/VARK and the LSI to be the ones that appeared with greater presence on the entire internet. When we focused solely on studies in Spanish, the most significantly relevant questionnaire was the CHAEA (the Honey-Alonso Learning Styles Questionnaire adapted into Spanish, Honey and Mumford (1982)), followed by the VAK/VARK. When we focused on all of the questionnaires analysed in this paper directly related to the LSs, but taking into consideration the type of search engine used, we found that on both types of search engines the relevant questionnaires were the LSI and the ILS (Felder-Soloman). We also found that the only difference between both types is that the VAR/ VARK appears as relevant in the generic search engine, but does not in the specific search engines. Of those four, the most relevant questionnaires were: The LSI, LSQ, VAK/VARK and the ILS (Felder-Soloman), only the last two have been validated by different authors. The ILS (Felder-Soloman) questionnaire measures learning preferences or tendencies, while the VAK/VARK questionnaire measures modalities in the perception of information. Table VII (Annex) contains a summary of the characteristics of both questionnaires. Both tools accord with the original semantics of the term “Learning Styles” (Gibson, 1969). The limitation found in this study entails focusing the bibliographic review exclusively on two languages, Spanish and English, although we do believe that they encompass most of the literature on the subject. The immediate vision of this study would be the validation of the ILS questionnaire in Spanish with the aim of using it to determine Learning Styles in primary and secondary education students and to continue delving deeper into teaching strategies, techniques and methods that adapt to said Learning Styles.
104 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
References
Abdollahimohammad, A. and Ja’afar, R. (2014). Learning style preferences of nursing students at two universities in Iran and Malaysia. Journal of educational evaluation for health professions, 11, 30, 1-5. doi: 10.3352/jeehp.2014.11.30 Abdollahimohammad, A. and Ja’afar, R. (2015). Associations of learning style with cultural values and demographics in nursing students in Iran and Malaysia. Journal of educational evaluation for health professions, 12, 42, 1-6. doi: 10.3352/jeehp.2015.12.42 Aiello, R. C., García, M. R. and Jaramillo, M. (2015). Determinación de los estilos de aprendizaje de estudiantes de 1er curso de Ingeniería Industrial y Electrónica de la Universidad Técnica del Norte. Ibarra. Ecuador: Journal of Learning Styles, 7, 14, 43-67. Al Hamdani, D. (2015). Exploring students’ learning style at a Gulf University: a contributing factor to effective instruction. Procedia- Social and Behavioral Sciences, 176, 124-128. doi: 10.1016/j. sbspro.2015.01.452 Alavi, S. S. and Makarem, J. (2016). Learning Style and Attitude toward Computer among Iranian Medical Students. Journal of Medical Education, 14, 1, 20-25. Al-Azawei, A., Parslow, P. and Lundqvist, K. (2015). A Psychometric Analysis of Reliability and Validity of the Index of Learning Styles (ILS). International Journal of Psychological Studies, 7, 3, 46-57. Aljaberi, N. M. (2015). University Students’ Learning Styles and Their Ability to Solve Mathematical Problems. International Journal of Business and Social Science, 6, 4, 152-165. Allinson, C. W. and Hayes, J. (1996). The Cognitive Style Index: A Measure of Intuition-Analysis For Organizational Research. Journal of Management Studies, 33, 119–135. doi: 10.1111/j.1467-6486. 1996. tb00801.x Alonso, C. M., Gallego, D. J. and Honey, P. (1994). Cuestionario Alonso– Honey de Estilos de Aprendizaje (CHAEA). Retrieved from https:// www.researchgate.net/publication/265265933_CUESTIONARIO_ HONEY-ALONSO_DE_ESTILOS_DE_APRENDIZAJE_CHAEA Alonso, C. M., Gallego, D. J. and Honey, P. (1999). Los estilos de aprendizaje. Procedimientos de diagnóstico y mejora. Bilbao: Ediciones Mensajero.
Revista de Educación, 381. July-September 2018, pp. 89-125 105 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Apter, M. J., Mallows, R. and Williams, S. (1998). The development of the Motivational Style Profile.Personality and Individual Differences, 24, 1, 7-18. Armstrong, S. J. and Qi, M. (2016). A reassessment of the factor structure of the Allison-Hayes Cognitive Style Index. Personality and Individual Differences, 101, 240-242. Backhaus, K. and Liff, J. P. (2007). Cognitive Style Index: Further investigation of the factor structure with an American student sample. Educational Psychology, 27, 1, 21-31. Barbosa, S. D., Gerhardt, M. W. and Kickul, J. R. (2007). The role of cognitive style and risk preference on entrepreneurial self-efficacy and entrepreneurial intentions. Journal of Leadership and Organizational Studies, 13, 4, 86-104. Barca, A. (1999). Manual del Cuestionario de Procesos de Estudio y Aprendizaje (CEPEA). A Coruña: Publicaciones de la Revista Galego- Portuguesa de Psicoloxia e Educación. Barch, J. (1996). Learning styles inventory. Leveroni Court. Novato, USA: Academic Therapy Publications. Barros, D. M. V. (2010). Estilos de uso do espaço virtual: Novas perspectivas para os ambientes de aprendizagem online. Journal of Learning Styles, 3, 6, 103-127. Baus, T. (2007). Los estilos de aprendizaje.” Retrieved from http://www. monografias.com/ trabajos12/losestils/losestils.shtml”. Bhagat, A., Vyas, R. and Singh, T. (2015). Students awareness of learning styles and their perceptions to a mixed method approach for learning. International Journal of Applied and Basic Medical Research, 5, 1, S58-S65. doi: 10.4103/2229-516X.162281 Biggs, J. (1985). Test review of Barsch Learning Style Inventory. In: J. V.Mitchell (Ed.), The ninth mental measurements yearbook (3-4). “Retrieved from http://marketplace.unl.edu/buros/” Biggs, J. (1987). Reflective thinking and school learning: An introduction to the theory and practice of metacognition (Item IO, Set 2. 14). Wellington, NZ: NZCER. Bothma, T. J. and de Boer, A. L. (2015). Reflecting on Diverse Teaching Methodologies for an Information Literacy Programme for Large Groups. The Third European Conference on Information Literacy (ECIL). Tallinn, Estonia, 19-22, October.
106 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Bunderson, C. V. (1994). The validity of the Herrmann Brain dominance instrument. In N. Herrmann (Ed.), The creative brain (364). Lake Lure, NC: Brain Books. Burnett, P. C. and Dart, B. C. (2000). The study process questionnaire: a construct validation study. Assessment and Evaluation in Higher Education, 25, 1, 93-99. doi: 10.1080/713611415 Çardak, Ç. S. and Selvi, K. (2016). The Construct Validity of Felder- Soloman Index of Learning Styles (ILS) for the Prospective Teachers. Gaziantep University Journal of Social Sciences, 15, 2, 675-693. Cheng, G. (2014). Exploring students’ learning styles in relation to their acceptance and attitudes towards using Second Life in education: A case study in Hong Kong. Computers and Education, 70, 105-115. Coffield, F., Moseley, D., Hall, E. and Ecclestone, K. (2004). Learning styles and pedagogy in post-16 learning. A systematic and critical review. London: Learning & Skills Research Centre. “Retrieved from http://www.leerbeleving.nl/wp-content/uploads/2011/09/learning- styles.pdf” Coloma, C. R., Manrique, L., Revilla, D. and Tafur, R. (2008). Estudio descriptivo de los estilos de aprendizaje de docentes universitarios. Revista de Estilos de Aprendizaje, 1, 1,124-142. Curry, L. (1990). One critique of the research on learning styles. Educational Leadership, 48, 50–56. De Boer, A. L., Bothma, T. and du Toit, P. (2011). Enhancing information literacy through the application of whole brain strategies. Libri, 61, 1, 67-75. doi: 10.1515/libr.2011.006 Díaz, M. A. C. and Aly, J. L. M. (2015). Estilos de aprendizaje en estudiantes universitarios: recursos informáticos como estrategia para su evaluación. UNAH INNOV@, 4, 33-39. Donche, V., Coertjens, L. and Van Petegem, P. (2010). Learning pattern development throughout higher education: A longitudinal study. Learning and Individual differences, 20, 3, 256-259. Dunn R., Dunn K. and Price G. (1975). Learning Style Inventory (LSI). Lawrence, Kansas: Price System. Dunn R., Dunn, K. and Price G. (1979). Learning Style Inventory (LSI) for Students in grades 3-12. Lawrence, Kansas: Price Systems Dunn, R. and Dunn, K. (1978). Teaching students through their individual learning styles: A practical approach. New Jersey: Prentice Hall.
Revista de Educación, 381. July-September 2018, pp. 89-125 107 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Dunn, R., Dunn, K. and Price, G. (1985). Manual: Learning Style Inventory. Lawrence, Kansas: Price Systems. Dunn, R., Dunn, K. and Price, G. (1989). Learning Style Inventory (LSI) for Students in grades 3-12. Lawrence, Kansas: Price Systems. Entwistle, N. (1988). Styles of learning and teaching. London: David Fulton. Escanero-Marcén, J. F., Soria, M. S., Guerra-Sánchez, M., and Silva, J. (2016). Comparación de los estilos de aprendizaje de los alumnos de medicina obtenidos con un nuevo cuestionario con los proporcionados por el cuestionario Honey-Alonso (CHAEA). FEM: Revista de la Fundación Educación Médica, 19, 1, 19-26. Felder, R. M. and Spurlin, J. (2005). Applications, reliability and validity of the index of learning styles. International Journal of Engineering Education, 21, 1, 103-112. Felder, R. M. and Soloman, B. A. (1997). Index of Learning Styles. “Retrieved from https://www.webtools.ncsu.edu/learningstyles/”. Fleming, N. D. (2001). Teaching and learning styles: VARK strategies. Christchurch, New Zealand: N.D. Fleming. Fleming, N. D. and Mills, C. (1992). Not another inventory, rather a catalyst for reflection. “Retrieved from http://digitalcommons.unl. edu/cgi/viewcontent.cgi?article=1245&context=podimproveacad” Fleming, N. D. and Mills, C. (2006). Learning Styles Again: VARKing up the right tree! Educational Developments, SEDA Ltd, 7, 4, 4-7. Fleming, N. D. (2012). Facts, Fallacies and Myths: VARK and Learning Preferences. “Retrieved from http://vark-learn.com/wp-content/ uploads/2014/08/Some-Facts-About-VARK.pdf”. Gamboa-Mora M. C., Briceño-Martínez J. J. and Camacho-González J. P. (2015). Caracterización de estilos de aprendizaje y canales de percepción de estudiantes universitarios, 31, 3, 509-527. “Retrieved from http://www.redalyc.org/pdf/310/31045567026.pdf” García, J. L, Santizo, J. A. and Alonso, C. M. (2009). Instrumentos de medición de estilos de aprendizaje. Review of learning styles, 4, 29, 3-21. Gargallo, B., Almerich, G., Suárez, J. M., García, E. and Garfella, P. R. (2012). Learning styles and approaches to learning in excellent and average first-year university students. European journal of psychology of education. 28, 4, 1361-1379. doi: 10.1007/s10212-012-0170-1
108 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Gibson, E. J. (1969). Principles of perceptual learning and development. NewYork: Appleton-Century Crofts. González-Peiteado, M. (2013). Los estilos de enseñanza y aprendizaje como soporte de la actividad docente. Journal of Learning Styles, 11, 11, 51-70. Grasha, A. F. (1996). Teaching with Style. A practical guide to enhancing learning by understanding teaching and learning style. Pittsburgh: Alliance Publishers. Gregorc, A. F. (1982). Gregorc Style Delineator: development, technical and administration manual. Columbia, CT: Gregorc Associates Inc. Herrmann, N. (1982). Herrmann Brain Dominance Instrument. Applied Services Herrmann, N. (1989). The creative brain. North Carolina: Brain Books, The Ned Herrmann Group. Hodgkinson, G. P. and Sadler-Smith, E. (2003). Reflections on reflections … on the nature of intuition, analysis and the construct validity of the Cognitive Style Index. Journal of Occupational and Organizational Psychology, 76, 279–281. Honey, P. and Mumford, A. (1982). Manual of Learning Styles. London: P. Honey. Hosford C. C. and Siders W. A. (2010). Felder-Soloman´s Index of Learning Styles: internal consistency, temporal stability and factor structure. Teach Learn Med. 22, 4, 298-303. doi: 10.1080/10401334.2010.512832. Huang, E. Y., Lin, S. W. and Huang, T. K. (2012). What type of learning style leads to online participation in the mixed-mode e-learning environment? A study of software usage instruction. Computers y Education, 58, 1, 338-349. Hughes, M., Hughes, P. and Hodgkinson, I. R. (2016). In pursuit of a ‘whole-brain’ approach to undergraduate teaching: implications of the Herrmann brain dominance model. Studies in Higher Education, 42, 12, 1-17. doi: 10.1080/03075079.2016.1152463 Jackson, C. J. (2002). Manual of the Learning Styles Profiler. Sydney: Cymeon Research. Joy, S. and Kolb, D. A. (2009). Are there cultural differences in learning style? International Journal of Intercultural Relations, 33, 1, 69-85. Kagan, J. (1966). Reflection-impulsivity: the generality and dynamics of conceptual tempo. Journal of Abnormal Psychology, 71, 1, 17-24.
Revista de Educación, 381. July-September 2018, pp. 89-125 109 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Ka-J, W. and Teo, A. (2016). Communication Strategy Use in an Oral Narrative Task among English Learners with Different Hemispheric Brain Dominance. LEARN Journal: Language Education and Acquisition Research Network, 9, 2, 188-198. Kalantari, M., Tahan, M. and Taraghi, A. (2016). Study of Innovation in Learning Styles of Students in Different Secondary School Branches. Journal of Psychology and Psychotherapy, 6, 274. doi: 10.4172/2161- 0487.1000274 Kappe, F. R., Boekholt, L., Den Rooyen, C. and Van der Flier, H. (2009). A predictive validity study of the Learning Style Questionnaire (LSQ) using multiple, specific learning criteria. Learning and Individual differences, 19, 4, 464-467. Kickul, J., Gundry, L. K., Barbosa, S. D. and Whitcanack, L. (2009). Intuition versus analysis? Testing differential models of cognitive style on entrepreneurial self-efficacy and the new venture creation process. Entrepreneurship Theory and Practice, 33, 2, 439-453. doi: 10.1111/j.1540-6520.2009.00298.x Knapp, T. R. (1998). Test review of Adaptive Style Inventory. In: J. C. Impara and B. S. Plake (Eds.), The thirteenth mental measurements yearbook. “Retrieved from: http://marketplace.unl.edu/buros/” Kolb, A. Y and Kolb, D. A. (2013). The Kolb Learning Style Inventory, 4.0. A comprehensive guide to the theory, psychometrics, research on validity and educational applications. Experience Based Learning Systems. “Retrieved from https://www.researchgate.net/profile/David_Kolb/ publication/303446688_The_Kolb_Learning_Style_Inventory_40_ Guide_to_Theory_Psychometrics_Research_Applications/ links/57437c4c08ae9f741b3a1a58/The-Kolb-Learning-Style-Inventory- 40-Guide-to-Theory-Psychometrics-Research-Applications.pdf”. ” Kolb, D. A. (1984). Experimental learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall. Kolb, D. A. (1999a) Learning Style Inventory, Version 3. Boston, MA: Training Resources Group. Kolb, D. A. (1999b) Learning Style Inventory Version 3: Technical Specifications. Boston, MA: Training Resources Group. Lau, W. W. F., Yuen, A. H. K. and Chan, A. (2015). Variable-Centered and Person-Centered Approaches to Studying the VARK Learning Style Inventory. Singapore: Springer.
110 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Leite, W. L., Svinicki, M. and Shi, Y. (2010). Attempted Validation of the Scores of the VARK: Learning Styles Inventory with Multitrait- Multimethod Confirmatory Factor Analysis Models. Educational and Psychological Measurement, 70, 323-339. Manolis, C., Burns, D. J., Assudani, R. and Chinta, R. (2013). Assessing experiential learning styles: A methodological reconstruction and validation of the Kolb Learning Style Inventory. Learning and individual differences, 23, 44-52. Martínez, G. and Manzo, S. (2013). Aplicación del modelo cuadrante cerebral de Herrmann y su relación con los estilos de aprendizaje. XIV Congreso virtual de psiquiatría.com Interpsiquis, 1-28 febrero. Martínez-Fernández, J. R., García-Ravidá, L., González-Veláquez, L., Gutiérrez-Braojos, C., Poggioli, L., Ramírez- Otálvaro, P. and Tellería, M. B. (2009). Inventario de Estilos de Aprendizaje en español. Documento interno del Grup de Recerca PAFIU. Barcelona: Universitat Autònoma de Barcelona. “Retrieved from http://grupsderecerca.uab.cat/pafiu/” Meneely, J. (2010). Educating adaptable minds: how diversified are the thinking preferences of interior design students? Journal of Interior Design, 35, 3, 21-32. doi: 10.1111/j.1939-1668.2010.01040.x Miller, M., De Clerck, J. P., Sorby, S. A., Roberts, L. M., Endres, W. J. and Hale, K. D. (2013). Meeting the NAE Grand Challenge: Personalized Learning for Engineering Students through Instruction on Megacognition and Motivation Strategies. 120th ASEE Annual Conference Proceedings, Atlanta, Georgia, USA. 23-26 June. Miller, M., Prabhakara, P. and Sorby, S. A. (2015). Reflection and Evaluation Data from e-Learning Modules on Learning Styles and Motivation. Proc. ASEE Annual Conference and Exposition, Seattle, WA, USA, 14- 17 June. Miller, M., Sorby, S. and De Clerck, J. (2015). E-Learning Modules for Improving Lifelong Learning Ability, Proc. ASEE Annual Conference and Exposition, Seattle, WA, USA, 14-17 June. Mortensen, C. J., Thoron, A. C. and Miot, J. K. (2015). Current learning styles of undergraduate animal-studies students in a 2-year and 4-year degree program. NACTA Journal, 59, 1, 75-80. Mutua, M. N. (2015). A correlation study between learning styles and academic achievement among secondary school students in Kenya (Tesis inédita de doctorado). Universidad de Nairobi. Kenia.
Revista de Educación, 381. July-September 2018, pp. 89-125 111 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Myers, I. B. and McCaulley, M. H. (1985). Manual: a guide to the development and use of the Myers-Briggs Type Indicator. Palo Alto, CA: Consulting Psychologists Press. Myers, I. B. and McCaulley, M. H. (1998). MTBI manual: a guide to the development and use of the Myers-Briggs Type Indicator. Palo Alto, CA: Consulting Psychologists Press. Njoku, J. N. and Abdulhamid, B. (2016). Preference of Learning Styles and its Relationship with Academic Performance among Junior Secondary School Students in Dutse Local Government Area, Jigawa State, Nigeria. International Journal of Education and Practice, 4, 3, 127-133. O’Connor, P. J., and Jackson, C. J. (2008). The factor structure and validity of the Learning Styles Profiler (LSP).European Journal of Psychological Assessment, 24, 2, 117–123. doi: 10.1027/1015-5759.24.2.117 O’Mahony, S. M., Sbayeh, A., Horgan, M., O’Flynn, S. and O’Tuathaigh, C. M. (2016). Association between learning style preferences and anatomy assessment outcomes in graduate-entry and undergraduate medical students. Anatomical sciences education, 9, 4, 391-399. doi: 10.1002/ase.1600 Ovington, L. A., Saliba, A. J. and Goldring, J. (2016). Dispositional Insight Scale: development and validation of a tool that measures propensity toward insight in problem solving. Creativity Research Journal, 28, 3, 342-347. Parker, J. A. (1985). Test review of Barsch Learning Style Inventory. In: J. V. Mitchell (Ed.), The ninth mental measurements yearbook. “Retrieved from http://marketplace.unl.edu/buros/” Pashler, H., McDaniel, M., Rohrer, D. and Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological Science in the Public Interest. 9, 3, 103-119. Penger, S., Tekavcic, M. and Dimovski, V. (2011). Comparison, validation and implications of learning style theories in higher education in Slovenia: an experiential and theoretical case. International Business and Economics Research Journal, 7, 12, 25-44. Peterson, K., DeCato, L. and Kolb, D. A. (2015). Moving and learning: Expanding style and increasing flexibility. Journal of Experiential Education, 38, 3, 228-244. Prien, E. P. (1998). Test review of Adaptive Style Inventory. In J. C. Impara and B. S. Plake (Eds.), The thirteenth mental measurements yearbook. “Retrieved from http://marketplace.unl.edu/buros/”
112 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Pupo, E. A. (2012). Los estilos de enseñanza, una necesidad para la atención de los estilos de aprendizaje en la educación universitaria. Journal of Learning Styles, 10, 10, 79-87. Quiñones, C. (2004). Metodología de Estrategia Enseñanza-Aprendizaje y Estilos de Aprendizaje. UMBRAL. Revista de Educación, Cultura y Sociedad, 6, 48-61. Reio, T. G. and Wiswell, A. K. (2006). An examination of the factor structure and construct validity of the Gregorc Style Delineator. Educational and Psychological Measurement, 66, 3, 489-501. Reynolds, C. R. (2003). Test review of Learning Style Inventory, Version 3. In B. S. Plake, J. C. Impara, and R. A. Spies (Eds.), The fifteenth mental measurements yearbook. “Retrieved from http://marketplace.unl.edu/ buros/” Riding, R. (1991). Cognitive Styles Analysis – CSA administration. Birmingham: Learning and Training Technology. Rodríguez, A. L., Tijerina, B. A. T. and García, J. L. G. (2016). Implementación del instrumento Quirontest para medir estilos de aprendizaje en estudiantes de pregrado en línea. Journal of Learning Styles. 9, 17, 240-267. Saarikoski, L., Salojärvi, S., Del Corso, D, and Ovcin, E. (2001). The 3DE: An Environment for the Development of Learner-Oriented Customised Educational Packages. International Conference on Information Technology Based Higher Education and Training (ITHET): Kumamoto, 4-6 July. Sadler-Smith, E., Spicer, D. P. and Tsang, F. (2000). Validity of the Cognitive Style Index: replication and extension. British Journal of Management, 11, 2, 175-181. doi: 10.1111/1467-8551.t01-1-00159 Schar, M. (2015). Pivot thinking: Predicting entrepreneurial intent among engineering students and faculty using problem solving style preference. Frontiers in Education Conference (FIE). El Paso, Texas, USA, 21-24 October. Shaikh, V. and Waychal, P. (2015). Experience of using felder–soloman index of learning styles. Proceedings of the International Conference on Transformations in Engineering Education, 369-378. doi: 10.1007/978-81-322-1931-6_43 Shinnick, M. A. and Woo, M. A. (2015). Learning style impact on knowledge gains in human patient simulation. Nurse Education Today, 35, 1, 63- 67. doi: 10.1016/j.nedt.2014.05.013
Revista de Educación, 381. July-September 2018, pp. 89-125 113 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Shukr, I., Zainab, R. and Rana, M. H. (2013). Learning styles of postgraduate and undergraduate medical students. Journal of College of Physicians and Surgeon Pakistan, 23, 1, 25-30. doi: 01.2013/JCPSP.2530. Shum, D. (2003). Test review of Learning Style Inventory, Version 3. In: B. S. Plake, J. C. Impara, and R. A. Spies (Eds.), The fifteenth mental measurements yearbook. “Retrieved from http://marketplace.unl.edu/ buros/” Singh, L., Govil, P. and Rani, R. (2015). Learning Style preferences among secondary school students. International Journal of Recent Scientific Research. 6, 5,3924-3928. Sternberg, R.J. (1999). Thinking styles. Cambridge: Cambridge University Press. Tracz, W. (2017). An exploratory study of Ontario college electrical engineering students learning styles. (Tesis inédita de doctorado). Universidad de Athabasca. Canadá. van Heerden, H., van Eck, E., Burger, M. and Khan, R. (2016). Learning Style Preferences of Undergraduate Quantity Surveying Students: Quantity Surveying. HASSACC-Human and Social Sciences at the Common Conference. 3-7 octubre. Velásquez, A. M. V., Ortiz, J. F. Z. and Rodríquez, A. L. (2016). La relación entre los estilos de aprendizaje y el rendimiento académico en matemáticas en alumnos de ciclo v de educación secundaria. Journal of Learning Styles, 9, 18, 70-94. Vermunt, J. D. (1996). Metacognitive, cognitive and affective aspects of learning styles and strategies: a phenomenographic analysis. Higher Education, 31, 1, 25–50. Villalobos, L., González, M., Muñoz, J., Ostoic, C. and Veliz, N. (2016). Estilos de aprendizaje de Honey-Alonso en estudiantes de ingreso universidad Antofagasta 2012-2013-2014-2015. V Congreso CLABES. Talca, Chile, 11-13 noviembre. Wang, J. and Mendori, T. (2015). The Reliability and Validity of Felder- Silverman Index of Learning Styles in Mandarin Version. Information Engineering Express International Institute of Applied Informatics, 1, 3, 1-8. Wilkinson, T., Boohan, M. and Stevenson, M. (2014). Does learning style influence academic performance in different forms of assessment?. Journal of anatomy, 224, 3, 304-308. doi: 10.1111/ joa.12126.
114 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Wismath, S., Orr, D. and Good, B. (2014). Metacognition: Student reflections on problem solving. Journal on Excellence in College Teaching, 25, 2, 69-90. Wismath, S., Orr, D. and Zhong, M. (2014). Student perception of problem solving skills. Trans formative Dialogues, 7, 3, 1-17. Wium, A. M., Pitout, H., Human, A. and du Toit, P. H. (2015). An analysis of thinking preferences across three health care disciplines. Innovations in Education and Teaching International, 1-9. Zywno, M. (2003). A Contribution to Validation of Score Meaning for Felder Soloman’s Index of Learning Styles. Annual Conference, Nashville, Tennessee, 22-25 June.
Contact address: Mª Luz Diago Egaña. Universidad Internacional de la Rioja. Facultad de Educación. Departamento de Didáctica de las Matemáticas y de las Ciencias Experimentales. Avenida de la Paz, 137. 26006, Logroño. (La Rioja). España. E-mail: [email protected]
Revista de Educación, 381. July-September 2018, pp. 89-125 115 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Annex
TABLE I. Keywords, instruments and acronyms used in searches on different databases. All the terms were used in two languages (Spanish and English).
KEYWORDS INSTRUMENTS ACRONYMS Inventory for students Barsch Learning Styles Inventory ASI Cognitive Style Index ASSIST Study and Learning Process Evaluation Ques- Academic performance BLSI tionnaire Cerebral hemispheres CEPEA Learning Styles according to the Question- and learning CESEA naire by Escanero and Soria Learning preferences CHAEA Honey-Alonso Learning Styles Questionnaire Learning strategies CSI Style of Use of the Virtual Space Learning styles GSD Gregorc Mind Styles Model and Style Delin- Learning styles and HBDI eator myths ILS Herrmann Brain Dominance Instrument Neuroscience KLSI Learning Style Index Perception channels LSI Learning Style Inventory Approaches and study LSP Kolb’s Adaptive Style Inventory skills LSQ Kolb’s Learning Style Inventory Cognitive styles LSS Learning Style Inventory Questionnaire MBTI Learning Style Profile Measurement instru- MSP Learning Style Questionnaire ment PNL Learning Style Scale Teaching styles SPQ Motivational Style Profile VAK Myers–Briggs Indicator VARK Quiron Test Study Process Questionnaire Barsch’s Test
Source: produced by the author(s) of this study
TABLE II. Characteristics of the CSI questionnaire used to measure “Learning Styles”, which, according to Coffield et al. (2004) meet the 4 minimum criteria.
Question- Refer- Dimen- Validated (yes Psychometric char- Construct naire name ence sions or no) acteristics YES (Sadler-Smith Intuitive et al. 2000; Cof- internal consistency, CSI (Cogni- Allinson Information style, field et al. 2004; test-retest reliability, tive Style and Hayes processing analytical Armstrong and Qi, construct validity and Index) (1996). preferences style 2016). predictive validity
Source: produced by the author(s) of this study
116 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
TABLE III. Characteristics of the questionnaires used to measure “Learning Styles”, which, accor- ding to Coffield et al. (2004) meet 2 to 3 minimum criteria.
Validat- Psychomet- Question- Reference Construct Dimensions ed (yes ric charac- naire name or no) teristics telic/paratelic – negativism/confor- Personality char- mity – acteristics (5 pairs autic mastery/sym- internal Apter, Mal- of motivational pathy YES consistency, MSP (Moti- lows and states and ten- alloic mastery/sym- (Coffield test-retest vational Style Williams dencies towards pathy et al. reliability and Profile) (1998) arousal, effort and arousal Avoidance/ 2004) predictive optimism/pes- Seeking validity simism) Optimism/pessimism Arousal/Effort
internal Directed towards consistency, ILS (Inven- meaning, directed YES Learning Strate- test-retest tory of Vermunt towards application, (Coffield gies and Ap- reliability, Learning (1996) directed towards et al. proaches construct va- Styles) reproduction and 2004) lidity (Coffield undirected et al. 2004) ASSIST (Approaches YES internal Learning Strate- and Study Entwistle Profound, superficial (Coffield consistency gies and Ap- Skills Inven- (1988) and strategic approach et al. and construct proaches tory for 2004) validity Students) YES HBDI (Her- (Coffield test-retest rmann’s Thinking styles/ Analytical, sequential, et al. reliability and Herrmann Brain Learning prefer- interpersonal and 2004); construct va- (1989) Dominance ences imaginative thinking Bunder- lidity (Coffield Instrument) son, et al., 2004) 1994) Myers and MBTI McCaulley Perceive/judge Feeling/ YES internal con- Personality char- (Myers- (1985). intuition, thinking/feel- (Coffield sistency and acteristics, Cogni- Briggs Type Myers and ing and Extraversion/ et al. test-retest tive structure Indicator) McCaulley introversion 2004) reliability (1998).
Source: produced by the author(s) of this study
Revista de Educación, 381. July-September 2018, pp. 89-125 117 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
TABLE IV. Characteristics of the questionnaires used to measure “Learning Styles”, which, accor- ding to Coffield et al. (2004) meet 1 minimum criterion.
Question- Validated Psychometric naire Reference Construct Dimensions (Yes or No) characteristics name LSQ YES (Coffield (Learning et al. 2004.); test-retest reli- style ques- Honey and Learning Active, reflective, NO (Kappe, ability tionnaire)/ Mumford Style Pref- theoretic and prag- Boekholt, CHAEA (1992) erences matic Den Rooyen, test-retest reli- (adapted and Van der ability into Span- Flier, 2009). ish) Coffield et al, (2004): predictive validity. Dunn and Dunn Leite et al. (2010): (1978); Modali- YES (Coffield Visual, auditory and reliability. They Dunn, ties in the et al. 2004; VAK/ kinaesthetic/Visual, advise the exer- Dunn, perception Leite, Svinicki VARK/PNL auditory, reading and cising of caution and Price, of informa- and Shi kinaesthetic in its research (1989)/ tion (2010). use regarding Fleming, the drawing up (2001) of items and the scoring of scale algorithms Prefer- YES (Coffield ences of Accommodating, et al. 2004) test-retest reli- LSI Ver- Kolb (1999a Experiential Diverging, Assimilat- NO (Reyn- ability sion 3 and b) Learning ing, Converging olds, 2003; ----- Styles Shum, 2003). Specific GSD channels Concrete sequential/ YES (Coffield (Gregorc’s for receiv- abstract, et al. 2004) Predictive validity Mind Styles Gregorc ing, pro- Random-summary NO (Reio Cronbach’s alpha Model and (1982) cessing and Sequential/ concrete and Wiswell, coefficients Style Delin- expressing random 2006) eator) information LSP Stable (Jackson’s SI (O’Connor, Jackson personality Initiator/Reasoner/ Learn- and Jackson, Factor structure (2002) character- Analyst/Implementer ing Styles 2008) istics Profiler)
Source: produced by the author(s) of this study
118 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
TABLE V. Tools used to measure “Learning Styles” that were not chosen by Coffield et al. (2004) with their metric characteristics.
Psycho- Validated Question- Dimen- metric Reference Construct (Yes or naire name sions character- No) istics SPQ (Study Process Ques- tionnaire) Superficial, Biggs (1987) Learning motiva- YES (Bur- CEPEA (Study profound Validity and tion and strate- nett and and Learning and achieve- reliability Barca (1999) gies Dart, 2000) Process Evalu- ment ation Ques- tionnaire) Barsch’s NO (Biggs, Test (Barsch Learning prefer- Visual, Audi- 1985; Barsch, (1996). ----- Learning Style ences tory, Tactile Parker, Inventory) 1985). Initiating, Experi- Measures the encing, YES (LSI extent to which KLSI 4 (Kolb Imagining, Version 4: Reliability, people change Learning Styles Kolb and Kolb Reflecting, Experi- internal and their Experiential Inventory, (2013) Analysing, ence Based external Learning Style Version 4) Thinking, Learning validity according to the Deciding, Styles, Inc) learning context Acting and Balancing Measures adap- http://learning- tive flexibility Acting, ASI (Kolb fromexperience. NO (Knapp, (how we adapt Evaluating, Adaptive Style com/tools/kolb- 1998; Prien, ------the LS to 4 dif- Thinking, inventory) adaptive-style- 1998) ferent learning Deciding inventory-asi/ contexts)
Revista de Educación, 381. July-September 2018, pp. 89-125 119 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Test-retest reliability, factor struc- ture, inter- YES (Zywno, nal validity, 2003; total cor- relation and cross-scale correlation Felder and Reliability Spurlin, and validity 2005; Sensory or Internal intuitive Hosford and consistency, Visual or Siders 2010; ILS (Index Learning prefer- temporal Felder and Solo- verbal of Learning ences or tenden- stability man, (1997) Active or Styles) cies and factor reflective Wang and structure Sequential Mendori, or Global 2015; Internal validity, Al-Azawei, cross-scale Parslow, and correla- Lundqvist, tion and 2015) construct NO validity (Çardak, and Reliability Selvi, 2016) and validity of moderate perceptions ----- Theoretic CESEA (Learn- (knowing), ing Styles ac- Escanero-Marcén, Dynamic cording to the Soria, Guerra- Learning styles (checking), NO ----- Questionnaire Sánchez and Silva (cognitive styles) Reflexive by Escanero (2016) (thinking), and Soria) Operative (doing)
120 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Participa- tory Style, Search and Research Style, Style of Use Style of use of Structuring of the Virtual Barros (2010) NO ----- virtual space and Planning Space Style Concrete Action and Production Style Analytical/ Preferences Global, De- when studying pendent/In- Rodríguez, Tijerina subjects online Cronbach’s Quiron Test dependent, YES and García (2016) or via the hybrid alpha Theoretical/ (online and class- Practical, Vi- based) modality sual/Verbal Perceptive, solitary, YES (Abdol- Cronbach’s Abdollahimoham- LSS (Learning Learning prefer- analytic, lahimoham- alpha mad and Ja’afar Styles Scale) ences competitive mad and coefficients, (2014) and imagina- Ja’afar, 2014) test-retest tive
Source: produced by the author(s) of this study
Revista de Educación, 381. July-September 2018, pp. 89-125 121 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
TABLE VI. Use of the different questionnaires in recent years.
Questionnaire Some post-2004 studies that have been used name (Barbosa, Gerhardt and Kickul, 2007; Kickul, Gundry, Barbosa and Whitcanack, CSI 2009; Ovington, Saliba and Goldring, 2016; Armstrong and Qi, 2016) (Meneely, 2010; De Boer, Bothma and du Toit, 2011; Martínez and Manzo, 2013; Bothma and de Boer, 2015; Schar, 2015; Wium, Pitout, Human, and du Toit, HBDI 2015; Hughes, Hughes and Hodgkinson, 2016; Ka-J and Teo, 2016; van Heerden, van Eck, Burger and Khan, 2016) (Penger, Tekavcic and Dimovski, 2011; Gargallo, Almerich, Suárez, García and Garfella, 2012; Shukr, Zainab, and Rana, 2013; Wilkinson, Boohan and Steven- LSQ son, 2014; Aljaberi, 2015; O’Mahony, Sbayeh, Horgan, O’Flynn and O’Tuathaigh, 2016; Villalobos, González, Muñoz, Ostoic and Veliz, 2016). (Penger, Tekavcic y Dimovski, 2011; Al Hamdani, 2015; Bhagat, Vyas and Singh, 2015; Tracz, 2017)/(Díaz and Aly, 2015; Lau, Yuen and Chan, 2015; Velásquez, VAK/VARK Ortiz, and Rodríquez, 2016)
Gergorc Style (Mortensen, Thoron and Miot, 2015) Delineator (GSD) (Miller, De Clerck, Sorby, Roberts, Endres and Hale, 2013; Wismath, Orr and Good, 2014; Wismath, Orr and Zhong, 2014; Miller, Prabhakara and Sorby, Barsch’s Test 2015; Miller, Sorby and De Clerck, 2015; Mutua, 2015; Singh, Govil and Rani, 2015; Alavi and Makarem, 2016; Njoku and Abdulhamid, 2016) ILS (Felder-Solo- (Huang, Lin and Huang, 2012; Cheng, 2014; Shaikh and Waychal, 2015) man) CESEA Escanero-Marcén, J. F., Soria, M. S., Guerra-Sánchez, M., and Silva, J. (2016) Reduced Kolb Learning Style (Manolis, Burns, Assudani and Chinta, 2013) Inventory (RLSI) KLSI (Peterson, DeCato and Kolb, 2015; Kalantari, Tahan and Taraghi, 2016) LSI (Donche, Coertjens and Van Petegem, 2010; Shinnick and Woo, 2015) LSS (Abdollahimohammad and Ja’afar, 2014; Abdollahimohammad and Ja’afar, 2015)
Source: produced by the author(s) of this study
122 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
TABLE VII. Summary of the instruments chosen to measure “Learning Styles”
Validated Questionnaire Con- Description of Psychometric Dimensions (Yes or name/Reference struct dimensions characteristics No) YES Test-retest (Zywno, reliability, fac- 2003; tor structure, internal validity, total correlation and cross-scale Felder and correlation Spurlin, 2005; Reliability and validity Hosford Sensory: Con- Learning and Siders Internal consis- ILS (Index of crete thinker, prefer- 2010; tency, temporal Learning Styles)/ Sensory/Intui- practical and ences or stability and fac- Felder and Solo- tive oriented towards tenden- Wang and tor structure man, (1997) facts and proce- cies Mendori, Internal valid- dures 2015; ity, cross-scale correlation Al-Azawei, and construct Parslow, validity and Lundqvist, Reliability and 2015) validity of mod- NO erate percep- (Çardak, tions and Selvi, 2016) ----- Intuitive: Concep- tual thinker, inno- vative, oriented towards theories and underlying meanings Visual: Prefer visual represen- tations of pre- Visual/Verbal sented material, such as pictures, diagrams and flow charts, etc. Verbal: Prefer written and spo- ken explanations
Revista de Educación, 381. July-September 2018, pp. 89-125 123 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
Active: Learn by Active/Reflec- trying things out, tive enjoy working with others Reflective: Learn by thinking things through, prefer working alone or with one or two people Sequential: Linear Sequential/ thinking process, Global learn in small incremental steps Global: holistic thinking process, systematic think- ers, learn in large leaps
Coffield et al, Prefer to see (2004): predic- information YES (Cof- tive validity. VAK/VARK/PNL/ presented in a field et al. Modalities Dunn and Dunn visual way instead 2004; Leite et al. in the per- (1978); Dunn, of in writing, for (2010): reliabil- ception of Visual Dunn, and Price, example, through Leite, ity. They advise informa- (1989)/Fleming, pictures, charts, Svinicki the exercising tion (2001) diagrams, circles, and Shi of caution in its arrows, printouts, (2010). research use etc. regarding the drawing up of items and the scoring of scale algorithms They learn best by listening to information (in oral presenta- tions, confer- Auditory ences, debates, etc.). They easily remember what they have heard and they express it verbally.
124 Revista de Educación, 381. July-September 2018, pp. 89-125 Received: 12-06-2017 Accepted: 23-02-2018 Diago Egaña, M. L., Cuetos Revuelta, M. J., González González, P. Analysis of the Learning Styles measurement tools
They prefer everything relat- ing to reading and writing, and they collect the infor- Reading/writ- mation shown as ing words. Learning materials that are mainly based on texts are greatly preferred by these students. They learn better through touching and doing. Practi- Kinaesthetic cal experience is important to them.
Source: produced by the author(s) of this study
Revista de Educación, 381. July-September 2018, pp. 89-125 125 Received: 12-06-2017 Accepted: 23-02-2018
Numerical magnitude processing and mathematics achievement1
Procesamiento de magnitudes numéricas y ejecución matemática
DOI: 10.4438/1988-592X-RE-2017-381-383
Josetxu Orrantia Sara San Romualdo Rosario Sánchez Laura Matilla Universidad de Salamanca David Muñez National Institute of Education. Center for Research in Child Development (Singapore) Lieven Verschaffel Katholieke Universiteit Leuven. Center for Instructional Psychology and Technology
Abstract Recent research suggests that individual differences in mathematics achievement relate to basic number processing skills such as the ability to process numerical magnitudes. A key issue in this recent field of research is whether non-symbolic magnitude processing predicts mathematics achievement. An alternative account posits that accessing non-symbolic magnitudes from symbolic numbers is the real predictor. In this study, we use a longitudinal predictive design. We expand extant research by analyzing the role of two-digit numbers. Fifty-two first grade students are involved in the study. Their numerical magnitude processing skills, both symbolic and non-symbolic (one-and two-
(1) This study was supported by a grant from the Ministry of Economy and Competitiveness (grant no. PSI2015-66802-P).
Revista de Educación, 381. July-September 2018, pp. 127-146 127 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
digit numbers), are evaluated at school entry. Two years later, we assess their mathematics achievement. Hierarchical regression analyses show that processing symbolic magnitudes of large quantities (two-digit numbers) is the most robust predictor for later math achievement. Results are interpreted in terms of their educational implications. Findings call attention on developing appropriate screening tools to identify students at risk for later mathematics difficulties.
Keywords: mathematics achievement, primary school, number processing, numerical magnitude processing, approximate number system
Resumen Recientes investigaciones sugieren que las diferencias individuales en ejecución matemática están relacionadas con las habilidades de procesamiento numérico básico, tales como la capacidad para procesar magnitudes numéricas. Una cuestión clave en este reciente campo de investigación es qué habilidades relacionadas con el procesamiento de magnitudes predicen la ejecución en matemáticas, el procesamiento de magnitudes no simbólicas o el acceso a esas magnitudes desde los números simbólicos. En este estudio extendemos esta investigación analizando el rol del tamaño de las magnitudes utilizando un diseño predictivo longitudinal. Cincuenta y dos participantes de 1º de Educación Primaria fueron evaluados en tareas de procesamiento de magnitudes numéricas, tanto simbólicas como no simbólicas con cantidades grandes y pequeñas, y dos años después se les evaluó en ejecución matemática. Los análisis de regresión jerárquica muestran que el procesamiento de magnitudes simbólicas de cantidades grandes (dos dígitos) es un predictor más robusto de la futura ejecución matemática que las demás medidas de procesamiento de magnitudes. Estos resultados se interpretan en términos de sus implicaciones educativas, específicamente en aspectos relacionados con la identificación temprana de estudiantes en riesgo de presentar dificultades en el aprendizaje de las matemáticas, algo prioritario en cualquier sistema educativo desde el punto de vista de la prevención.
Palabras clave: ejecución matemática, educación primaria, procesamiento numérico, procesamiento de magnitudes numéricas, sistema numérico aproximado
Introduction
The study of individual differences in mathematics has generated an important body of research aiming to uncover factors that explain such
128 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement differences. A variety of domain-general factors (e.g., working memory, executive functions, processing speed) and domain-specific factors have been suggested (LeFevre, Wells, & Sowinski, 2016). Among the later, the ability to process numerical magnitudes has attracted a wealth of research in the last ten years (see Lyons & Ansari, 2015, for a review). It is widely acknowledged that such ability to process and represent numerical magnitudes relates to math achievement longitudinally. This study aims to further explore that effect. Specifically, the objective of this research is to investigate whether children’s individual differences in numerical magnitude processing and representing skills at school entry predict arithmetic achievement two years later. The suggested timeframe is relevant, as children at school entry do not master the arithmetic skills that are evaluated two years later. The following paragraphs provide an updated review of the most recent research in the field.
Numerical magnitude processing and mathematics achievement
There is evidence that humans (as well as other non-human animals) are equipped with an innate mechanism that allows representing and processing non-symbolic numerical magnitudes (Dehaene, 2011). This mechanism is known as the approximate number system (ANS), a primitive non-verbal representational system that does not depend on explicit teaching, and allows individuals to discriminate non-symbolic quantities (Feigenson, Dehaene, and Spelke, 2004). Discrimination acuity is governed by Weber’s law. Thus, discrimination between two magnitudes depends on their relative ratio rather than on absolute difference. The most widely used task to measure ANS acuity is non- symbolic magnitude comparison (e.g., Libertu, Feigenson and Halberda, 2011). In this task, participants are presented with two arrays of dots, and prompted to decide which array has more dots. Response time and accuracy decrease as the ratio or distance between magnitudes increases. It is assumed that numerical magnitudes are represented in an approximate way (i.e., as a Gaussian distribution around a specific magnitude) on a hypothetical mental number line. Thus, the effects of ratio and distance are due to greater representational overlaps when magnitudes are closer on that mental line. In this sense, distance and ratio effects are widely acknowledged as acuity indicators of numerical
Revista de Educación, 381. July-September 2018, pp. 127-146 129 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
magnitude representation. Importantly, the size of these effects decreases with age, leading to increasingly accurate representations (Halberda & Feigenson, 2008). Although ANS representations are approximate and differ from symbolic representations—exact representations of quantities—, it has been suggested that ANS may be the foundation on which symbolic numerical skills are built (Dehaene, 2011). Symbolic numerical magnitudes would be projected onto pre-existing ANS representations when symbols are learned to represent numbers (Barth, Starr, & Sullivan, 2009; Lipton & Spelke, 2005; Mundy & Gilmore, 2009; but see Reynvoet & Sasanguie, 2016, for an alternative explanation). Indeed, distance and ratio effects also occur in symbolic magnitude comparison tasks wherein subjects are required to compare Arabic digits or verbal numbers. This finding points towards a representational overlap in the mental number line as magnitude representations are activated when symbolic numbers are processed. Whilst it is not clear yet how symbols are projected onto ANS representations (see Leibovich & Ansari, 2016, for a review of this issue), it has been suggested that ANS plays an important role in learning symbolic mathematics. A significant number of studies have found that individual differences in ANS acuity relate to mathematics achievement. Two recent meta-analyses (Chen & Li, 2014; Fazio, Bailey, Thompson, & Siegler, 2014) have shown a moderate, but significant relationship between non-symbolic numerical magnitude processing and arithmetic achievement. However, results are less conclusive when both non- symbolic and symbolic magnitude processing skills are jointly analyzed. For example, in a narrative review, De Smedt, Noël, Gilmore and Ansari (2013) reported that only 44% of empirical studies analyzed in that review had a significant relationship between maathematics achievement and non-symbolic numerical magnitude processing. Importantly, 76% of the studies had a significant association with symbolic numerical magnitude processing. Similarly, another recent meta-analysis that included both non-symbolic and symbolic numerical magnitude comparison measures (Schneider et al., 2017), revealed that the effect size was significantly larger for the association between mathematics achievement and symbolic magnitude comparison. In sum, whilst extant research acknowledges that numerical magnitude processing can be foundational for mathematics learning, findings remain inconclusive. It is unknown whether non- symbolic, symbolic, or both numerical magnitude processing skills influence mathematic achievement (Orrantia et al., 2017).
130 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
In this context, it is worth noting a limitation of extant research. Most studies have included one-digit magnitude comparison tasks (i.e., Arabic numbers 1 to 9) (e.g., Xenidou-Dervou, Molenaar, Ansari, van der Schoot, & van Lieshout, 2017), and have sidelined that the development of numerical magnitude knowledge implies increasingly accurate representations of a widening range of numbers on a hypothetical mental number line (see The Integrated Theory of Numerical Development, Siegler, 2016). Indeed, children with relatively accurate representations of one-digit numbers may have less accurate representations of two-digit numbers. This has been reported on symbolic magnitude comparison tasks (e.g., Ashkenazi, Mark-Zigdon, & Henik, 2009) and number line tasks. In number line tasks, participants are required to estimate the place of a number on a bounded number line (i.e., 0-10 or 0-100) (e.g., Bertelletti, Lucangeli, Piazza, Dehaene, & Zorzi, 2010). Performance on this task would reflect participant’s ability to translate symbolic numbers onto non-symbolic magnitude representations. Thus, it is important to analyze to what extent two-digit magnitude processing skills at school entry relate to arithmetic achievement since, firstly, two-digit numbers are usually introduced at that point; and secondly, one-digit arithmetic activities are scarce from that point onwards. To date, there is only one study looking at this particular relationship in children. Xenidou- Dervou et al. (2017)2 run a longitudinal study wherein symbolic and non- symbolic magnitude comparison skills are factored with number size (1-9 vs. 6-70). The study span the kindergarten years and first year of formal education. Math achievement is assessed at the end of K-2. Their findings indicate that each measure follows a different evolutionary trajectory. More importantly, two-digit magnitude comparison is the most robust predictor of math achievement (beyond domain-general factors such as intelligence). It is worth noting that their magnitude comparison measures were based on accuracy (percentage of errors) and response times (RT), which may reflect how accurately ANS representations are accessed from symbolic numbers or, alternatively, how efficiently individuals process symbolic numerical information (De Smedt et al., 2013). Therefore, it is important to analyze whether other measures that explicitly tackle participants’ access to magnitude representations—numerical distance effect—are predictors of mathematics achievement.
(2) Linsen, Verschaffel, Reynvoet, and De Smedt (2014) also analyzed the relationship between numerical magnitude processing and mathematics achievement with small and large quantities, but their Study—cross-sectional—was run with older children.
Revista de Educación, 381. July-September 2018, pp. 127-146 131 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
The aim of the current study is to analyze whether children’s non- symbolic and symbolic magnitude processing and representing skills at school entry predict mathematics two years later. Additionally, it is explored whether that effect depends on number size. To that end, and in line with extant literature, magnitude comparison tasks (symbolic and non-symbolic) with small and large quantities are considered. Two types of measures are taken from each task, task performance or efficacy in numerical magnitude processing—error rate and RT—, and other measure reflecting magnitude representation accuracy. Combining different types of measures in the same study would provide valuable information on how individual differences in representing and processing numerical magnitudes impact math achievement. To investigate to what extent children’s numerical magnitude processing and representing skills explain math achievement beyond domain-general factors, a test of general cognitive skills is also administered along with a computerized processing speed task. Since the association between numerical magnitude processing and math achievement depends on how mathematics is defined (ranging from general math achievement measures to specific measures such as arithmetic skills; see Schneider et al, 2017), two mathematics measures are considered—a standardized test of elementary arithmetic skills (problem solving and mental calculation) and a task assessing children’s conceptual knowledge of additive properties.
Method
Participants
Data from the current study were drawn from an ongoing large-scale longitudinal study examining the interplay of symbolic and non-symbolic magnitude processing skills. Fifty-five first grade studentsM ( = 75 months, SD= 4 months) participated in the study. Sampled children were from two classes of the same school. The school was located in a middle-level socioeconomic area in Salamanca. Three participants were excluded for not completing all tasks. Thus, statistical analyses were performed on data from 52 students (24 females and 28 males). Parents provided informed written consent. Experimental procedures were approved by Human Experimentation Ethics Committee of the University of Salamanca (Spain)
132 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
Instruments
Large non-symbolic magnitude comparison (LNSC)
This is the most widely used task to measure ANS acuity. Each trial consisted of the presentation of two arrays of dots on both sides of a computer screen (15 inches). Participants were asked to indicate which array showed more dots by pressing a key, S (left) or L (right). The number of dots in the arrays varied from 6 to 60 points, with each pair depicting a ratio of .50, .66 and .75. The test consisted of 90 trials (30 per ratio) presented in two blocks (45 per block). Five practice trials were presented at the beginning of each block. Each trial began with a fixation point (1000 ms), followed by two arrays of dots that remained on screen for a limited time (1500 ms) to avoid counting. Half of trials were positively correlated (i.e., items in less numerous arrays were smaller and occupied a smaller area than those in more numerous arrays), and half of trials were negatively correlated (i.e., items in less numerous arrays were bigger and occupied a larger area than those in more numerous arrays).
Two-digit symbolic magnitude comparison (2SC)
In this task participants were asked to compare an Arabic number displayed in the middle of a computer screen against a fixed reference number (55) by pressing a key (L: displayed number is larger than reference number; S: smaller than reference number). All numbers between 31 and 79 (except reference number 55) were presented twice. Ninety-six trials were administered in two blocks (45 per block). Five practice trials were presented at the beginning of each block. Each trial began with a fixation point (1000 ms). Numbers remained on screen until participant’s response.
Small non-symbolic magnitude comparison (SNSC)
This task was similar to LNSC. Arrays of dots were replaced with arrays of filled squares. The number of squares in each array ranged from 1 to 9 squares, with each pair of arrays depicting distances ranging from 1
Revista de Educación, 381. July-September 2018, pp. 127-146 133 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
to 6. The test consisted of seventy-two trials presented in two blocks (36 trials per block). Five practice trials were presented at the beginning of each block.
One-digit symbolic magnitude comparison (1SC)
This task was similar to SNSC. Arrays of filled squares were replaced with the corresponding Arabic numbers.
Mathematics achievement
Children completed a standardized measure of elementary arithmetic skills (Arith), the Numerical Calculation and Numerical-Verbal Problems tests of the Differential and General Aptitude Battery (BADyG; Yuste, Yuste, Martínez, & Galve, 2012). Additionally, a non-standardized arithmetic word problem-solving task was administered—mental calculus (MC). In this task, the fluency with which magnitude information is available from numbers is important. The task consisted of 12 one-step Change 1, 2 and 3 word problems (according to Riley, Greeno, & Heller, 1983). Word problems were rephrased so questions were always stated in the first sentence of each problem. Simulating the action described in the problem (Brissiaud & Sander, 2010) allowed solving half of problems. For instance, in the word problem “How many marbles does Peter have now, if at the beginning of a game he had 42 marbles and then he lost 3 marbles?” children can simulate the action and directly subtract from the minuend. The remaining problems required children to activate their conceptual knowledge of arithmetic principles (i.e., commutability, inverse reference between addition and subtraction) to solve the problems. For instance, in the word problem “How many marbles Peter has now, if at the beginning of a game he had 42 marbles and then he lost 39 marbles” a direct simulation of the action would involve a high cognitive cost (Brissiaud & Sander, 2010). In contrast, the activation of strategies based on inverse reference would allow children to quickly determine how many marbles must be added to the subtrahend in order to reach the minuend. To ensure the use of these strategies, response time was limited to a maximum of 6
134 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement seconds. Furthermore, one of the terms/numbers (or both) was between 30 and 70 (the operation always involved a change of tens), and the other number or the distance between the two large numbers was 3 or 4.
Control tasks
Intellectual ability The ICCE Intelligence Test (Yuste, Franco, & Palacios, 2013) was used. The test is a general intelligence measure that includes verbal, numerical, spatial, and logical reasoning factors.
Processing speed (PS) This task was included to control for individual differences in reaction time when children were asked to press a key (computerized tasks). Two squares (one black and one red) were presented on both sides of a computer screen. Participants had to indicate where the red square was by pressing a key as quickly as possible, S (left) or L (right). The task consisted of 20 trials.
Procedure and Design
Computerized tasks were administered individually in a quiet room at the school. Presentation of stimuli and data collection were controlled with Cedrus SuperLab™ software. Computerized tasks along with other tasks not included in the current study were administered during the first year of formal school (April to May). Scores on the intelligence test were provided by the school. Mathematics achievement measures were collectively administered two years later. In line with the taxonomy of Ato, López and Benavente (2013), the current correlational study involved an associative strategy exploring the relationships between variables. Specifically, it was longitudinal predictive design, in which we analyzed a single sample by taking measurements at two different time points. Data were analyzed with SPSS. Repeated measures ANOVA were conducted to evaluate ratio and distance effects. Subsequently, Pearson correlations and hierarchical regression analyses were run to investigate the effects of predictive variables on criterion variables.
Revista de Educación, 381. July-September 2018, pp. 127-146 135 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
Results
Analyses involving LNSC were conducted on two different measures, accuracy rate (i.e., task performance), and Weber’s fraction which is an index that measures precision of the non-symbolic magnitude representation or ANS acuity (see procedure in Halberda & Feignson, 2008). Larger Weber’s fractions reflect lower ANS acuity. Descriptive information is presented in Table 1. Regarding 2SC, scores were firstly grouped into 6 different slots. Each slot related to a distance range (e.g., distance 1 included numbers 51-54 and 56-59, which were 1-to-4 units from 55; distance 2 included numbers 47-50 and 60-63, and distance 6 included numbers 31-34 and 76-79). Then, adjusted response times (adjRT) were calculated for each slot. AdjRTs combined speed and accuracy to control for potential speed-accuracy trade-offs (response times were divided by proportion of hits). Two measures were taken, one for task performance (mean adjRTs), and another measure reflecting precision of the magnitude representation—size of the distance effect (DE). DEs were computed by calculating individual slopes on the basis of a linear regression in which adjRTs were predicted by distance (with steeper slopes indicating larger distance effects). It is assumed that the slope should be negative, because the greater the distance, the smaller the RT. A steeper slope would suggest a less precise magnitude representation. Similar task performance (mean of adjRTs) and representational precision measures (distance effect) were calculated for SNSC and 1SC. Task performance scores would relate to children’s magnitude processing skills whereas precision measures would relate to magnitude representation skills.
136 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
TABLE I. Mean RT (and standard deviation) in milliseconds, and accuracy rates (proportion of hits) per distance and ratio for tasks addressing numerical magnitude.
DISTANCE 1 2 3 4 5 6 1731 1523 1430 1419 1404 1350 2SC RT (443) (415) (418) (425) (389) (419) AC .76 (.16) .82 (.18) .86 (.16) .88 (.13) .88 (.14) .91 (.10) 1284 1210 1155 1164 1132 1091 1SC RT (323) (308) (257) (310) (286) (276) AC .88 (.12) .90 (.10) .94 (.06) .95 (.05) .95 (.05) .97 (.03) 1332 1251 1148 1121 SNSC RT 159 (249) 158 (250) (360) (363) (287) (237) AC .80 (.14) .89 (.11) .92 (.09) .95 (.08) .97 (.07) .97 (.07) RATIO 0.50 0.66 0.75 LNSC AC .76 (.12) .65 (.10) .62 (.10)
Note: 2SC = Two-digit symbolic magnitude comparison; 1SC = One-digit symbolic magnitude comparison; SNSC = Small non-symbolic magnitude comparison; LNSC = Large non-symbolic magnitude comparison.
We conducted separate repeated measures analyses of variance in order to verify ratio and distance effects on each magnitude comparison task. Ratio (3 levels: 0.50, 0.66 and 0.75) and distance (6 levels: distance 1 to 6) were within-subjects factors. As expected, the analyses revealed 2 a ratio effect in the LNSC task, F (2,102) = 60.82, p <. 0001, ηp =. 54, indicating that the proportion of hits decreased with increasing ratios (see Table 1). Similarly, the analyses also revealed a distance effect in the 2 remaining tasks [F (5,255) = 20.09, p <. 0001, ηp =. 28; F (5,255) = 19.31, p 2 2 <. 0001, ηp =. 27; and F (5,255) = 62.06, p <. 0001, ηp =. 55; 2SC, 1SC, and SNSC, respectively], which suggested that adjRT decreased as distance increased (see Table 1). Similar results have been systematically reported in previous studies. Pearson correlation coefficients (Table 2) were calculated to analyze the relationship between measures of magnitude processing and mathematics achievement. No significant correlation was revealed between LNSC measures (task performance and representational precision) and mathematics achievement. SNSC (representational precision) only
Revista de Educación, 381. July-September 2018, pp. 127-146 137 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
correlated with standardized measures of math achievement (arithmetic achievement). The remaining measures significantly correlated with both measures of mathematics achievement, arithmetic and MC. Higher scores in math measures related to shorter RTs and smaller distance effects.
TABLA II. Pearson correlations between magnitude-related measures and mathematics achieve- ment (arithmetic, and mental calculus—MC)
1 2 3 4 5 6 7 8 9 10
1 Arithmetic _ 81** .08 -.08 -.57** .45** -.38** .38** -.48** .34* 2 MC -.01 -.20 -.57** .48** -.20 .35* -.34* .35* 3 LNSC (DE) _ .83** -.02 -.18 -.24 .19 -.14 -.05 4 LNSC (w) _ .09 -.18 -.17 .02 .04 -.17 5 2SC (Eff) _ -.29* .41** -.46** .50** -.54** 6 2SC (DE) _ -.18 .03 -.14 .07 7 SNSC (Eff) _ -.70** .55** -.31* 8 SNSC (DE) _ -.48** .48** 9 1SC (Eff) _ -.40** 10 1SC (DE) _
Note: 2SC = Two-digit symbolic magnitude comparison; 1SC = One-digit symbolic magnitude comparison; SNSC = Small non-symbolic magnitude comparison; LNSC = Large non-symbolic magnitude comparison; DE = distance effect; Eff = task performance. * p < .05 * p < .01
Finally, two hierarchical regression analyses (Table 3) were run to explore the contribution of each numerical magnitude measure to variance in mathematics achievement. These analyses were run separately for MC and arithmetic. Step 1 included domain-general measures—intelligence and processing speed—to control for general cognitive skills. In Step 2, magnitude-related measures that showed a significant correlation with math achievement and MC were entered into the regression analyses.
138 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
TABLA III. Hierarchical regression analyses with math achievement (Arithmetic and MC)
Arithmetic MC
Step Predictor ß t R2 Predictor ß t R2
1 Intelligence .22 1.81a .29 Intelligence .32 2.69* .35 PS -.07 -.43 PS .06 .44 2 SNSC (Eff) -.02 -.10 .52 SNSC (DE) .10 .79 .54 SNSC (DE) .09 .55 1SC (Eff) -.10 -.80 1SC (Eff) -.26 -1.88b 1SC (DE) .06 .48 1SC (DE) -.01 -.08 2SC (Eff) -.31 -1.77c 2SC (Eff) -.20 -1.12 2SC (DE) .26 2.22* 2SC (DE) .26 2.19*
Note: 2SC = Two-digit symbolic magnitude comparison; 1SC = One-digit symbolic magnitude comparison; SNSC = Small non-symbolic magnitude comparison; LNSC = Large non-symbolic magnitude comparison; DE = distance effect; Eff = efficacy. * p < .05 a p = .08 b p = .06 c p = .08
In the first hierarchical regression analysis (arithmetic achievement as DV), the full model accounted for 52% of the variance, F (8, 43) = 5.72, p <. 0001. Magnitude measures accounted for 23% of the variance above and beyond the effect of control variables, F (6, 43) = 3.34, p <. 001. However, only the DE in 2SC was a significant predictor. Task performance in 1SC approached the statistical levels of significance. In a second hierarchical regression analysis (MC as DV), the full model explained 54% of variance, F (7,44) = 7.43, p <. 0001, and magnitude measures accounted for 19% of variance above and beyond the effect of control variables, F (5, 44) = 3.63, p <. 001. In this case, again, the DE in 2SC was the only significant predictor. 2SC task performance approached the statistical levels of significance. Given the moderate to high correlations between some measures, a collinearity analysis was run to detect potential multicollinearity issues. Variance inflation factors (VIF) were within acceptable range (all VIFs < 3).
Revista de Educación, 381. July-September 2018, pp. 127-146 139 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
Discussion and Conclusions
Recent research has revealed that children’s ability to process and represent numerical magnitudes relates to mathematics achievement. Nevertheless, there exists controversy about the role that symbolic and non-symbolic magnitude processing skills play in that relationship. One limitation of extant investigations is that most studies have used tasks involving small quantities (one-digit). That is an important caveat when symbolic magnitude comparison skills are assessed. Since children’s knowledge of symbolic numerical magnitudes expands from small numbers to large numbers (Siegler, 2016), it is necessary to explore whether a relation between numerical magnitude processing and mathematics achievement holds when large quantities are used. This is particularly relevant at school entry, as children are then required to understand and use two- digit numbers. The current study looked at this issue. Magnitude-related tasks involved non-symbolic and symbolic magnitude comparison of small and large quantities. The contribution of each task to later mathematics achievement was explored. Overall, the results showed that children’s ability to process two-digit symbolic magnitudes at the beginning of formal schooling related to mathematics achievement longitudinally. That effect was independent of domain-general skills such as intellectual ability and processing speed. However, that relation was largely affected by how magnitude processing was measured. Furthermore, different conceptualizations of math achievement (arithmetic vs. mental calculus) revealed slightly different patterns of results. Regarding the contribution of non-symbolic and symbolic numerical magnitude to children’s later mathematics, results showed that non- symbolic magnitude processing of large quantities did not correlate with math achievement. Although the effect of non-symbolic magnitude processing of small quantities correlated significantly with arithmetic achievement, that effect vanished once domain-general factors and symbolic magnitude processing skills entered into the regression equation. These findings support recent research that suggests that only symbolic numerical magnitude knowledge predicts variance in mathematics achievement when both symbolic and non-symbolic magnitude processing measures are jointly explored (e. g., Lyons, Price, Vaessen, Blomert, & Ansari, 2014; Sasanguie, Göbel, Moll, Smets, & Reynvoet, 2013). However, results of the current study go beyond previous findings
140 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement by demonstrating that two-digit symbolic magnitude processing skills at school entry is the most robust predictor of later math achievement. This result is consistent with Xenidou-Dervou et al. (2017). Nevertheless, our findings provide new insights. Children’s symbolic processing accuracy and speed (task performance) was not a significant predictor. In contrast, findings revealed that the distance effect was a robust predictor. High achievers showed smaller distance effects. Since smaller distance effects characterize acuity in mapping Arabic numerals onto underlying magnitudes, our results suggest that children’s understanding of numerical magnitudes plays an important role in learning mathematics. It is worth noting that Linsen, Verschaffel, Reynvoet, & De Smedt (2014) also found an association between magnitude processing of large quantities and mathematics achievement. However, that relation was only linked to participants’ performance (i.e., accuracy and speed), whereas the distance effect did not have a unique role in explaining mathematics achievement. Given that their data were drawn from a cross-sectional study conducted with participants of a higher educational level (third grade), findings suggest that children’s awareness of numerical magnitudes of two-digit numbers at school entry translates into efficiency in processing two-digit numbers later on. Since children usually start learning two-digit numbers when they enter formal school, a lack of performance-related effects in our study is feasible. Interestingly, that finding calls attention on the “developmental” sensitivity of numerical magnitude measures to assess later risks. In a similar vein, the current study also highlights how different measures of math achievement produce different patterns of results. Whilst the analyses revealed that a significant predictor of both, arithmetic and mental calculus was the size of the distance effect in two-digit symbolic magnitude comparison, task performance measures in one- digit symbolic magnitude comparison approached the standard levels of significance when they were correlated with arithmetic achievement. A possible explanation is that the standardized test that was used to assess mathematics achievement included a numerical calculation factor that focuses on one-digit problems. Vanbinst, Ghesquière, & De Smedt (2015) reported similar findings with children starting formal school— performance on symbolic magnitude comparison of small quantities predicted later achievement in a one-digit calculation task. In our study, mental calculation is assessed on two-digit arithmetic word problems.
Revista de Educación, 381. July-September 2018, pp. 127-146 141 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
Hence, it is not surprising that two-digit symbolic magnitude processing appeared as sole predictor, probably because a flexible use of arithmetic principles —such as the inverse reference principle—, requires a good understanding of the magnitudes of the numbers involved (Linsen et al., 2014). The current study has some limitations, which mainly relate to sample size and call for further research to explore the effects that approached standard levels of significance. Therefore, findings should be interpreted with caution. In any case, the sample was homogeneous, as two classes were selected from the same school. Thus, both groups were under similar teaching methods and identical curricula. Furthermore, results of the study were consistent with previous findings. Finally, some educational implications may be suggested. Given that a good understanding of numbers (in terms of their numerical magnitude) has been found to impact later mathematics achievement, developing assessment tools that tap on basic numerical processes such as those described in the current study may be deemed as necessary. Screening tools may contribute to early identification of students at risk of learning difficulties in mathematics—a priority in any educational system. Certainly, this approach is inspired by an ever growing and relevant development of similar tools in the reading domain. In that particular domain, another basic competence—phonological awareness—underlies individual differences in learning to read (Melby-Lervag, Lyster, & Hulme, 2012), and plays an important role in detection of at-risk students. Indeed, that has fostered a wide variety of tests that include measures related to that important competence. Despite heighten relevance of numerical magnitude processing skills in mathematics achievement, actual assessment measures do not usually list this numerical competence among to-be-assessed aspects. Furthermore, when that component is included, the number of items is rather small and there is no timing constraints. Nevertheless, the majority of the studies (including the present study) that analyzed the relevance of numerical magnitude processing skills in mathematics achievement focused on computerized paradigms, which may not always be suitable for quick application in the classroom and require specialized equipment and software (Nosworthy, Bugden, Archibald, Evans, & Ansari, 2013). In this context, Nosworthy et al. (2013) designed a timed (two minutes) pen-and-paper task to evaluate children’s ability to compare numerical magnitudes of small quantities (1
142 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement to 9). The authors found that task performance related to mathematics achievement. This finding suggests that similar tasks including large, double-digit numbers may be designed. This type of group-administered screening shows some advantages, such as rapid application, ecological validity, and minimal instruction. These features may ease potential implementations aimed at early detection of at-risk students (Brankaer, Ghesquière, & De Smedt, 2017). In addition to the above mentioned implications, the results of this study also have implications for educational practice. Since an adequate understanding of numerical magnitudes predicts mathematics achievement, reinforcing learners’ numerical magnitude processing skills in preparation for more complex mathematical contents may be a valuable learning tool. Although research in this field has not yielded conclusive results yet (De Smedt et al., 2013; Schneider et al., 2017), some mathematics curricula have already incorporated magnitude-related activities as part of early mathematics teaching. A paradigmatic example of this proposal is a Singapore publishing project, My Pals are here3, aimed to reinforce students’ ability to process numerical magnitudes throughout activities that connect numeric symbols with their meanings.
References
Ashkenazi, S., Mark-Zigdon, N., & Henik, A. (2009). Numerical distance, effect in developmental discalculia. Cognitive Development, 24, 387- 400. Ato, M., & López, J., & Benavente, A. (2013). Un sistema de clasificación de los diseños de investigación en psicología. Anales de Psicología, 29, 1038-1059. Barth, H., Starr, A., & Sullivan, J. (2009). Children’s mappings of large number words to numerosities. Cognitive Development, 24, 248-264. Bertelletti, I., Lucangeli, D., Piazza, M., Dehaene, S., & Zorzi, M. (2010). Numerical estimation in preschoolers. Developmental Psychology, 46, 545–551.
(3) This Project has been developed by the Marshall Cavendish publishing house for the most used textbooks in Singapore.
Revista de Educación, 381. July-September 2018, pp. 127-146 143 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
Brankaer, C., Ghesquière, P., & De Smedt, B. (2017). Symbolic magnitude processing in elementary school children: A group administered paper-and-pencil measure (SYMP Test). Behavior Research Methods, 49, 1361-1373. Brissiaud, R., & Sander, E. (2010). Arithmetic word problem solving: a Situation Strategy First framework. Developmental Science, 13, 92-107. Chen, Q., & Li, J. (2014). Association between individual differences in nonsymbolic number acuity and math performance: A meta-analysis. Acta Psychologica, 148, 163–172. Dehaene, S. (2011). The number sense: How the mind creates mathematics. New York: Oxford University Press. De Smedt, B., Noël, M. P., Gilmore, C., & Ansari, D. (2013). How do symbolic and nonsymbolic numerical magnitude processing skills relate to individual differences in children’s mathematical skills? A review of evidence from brain and behavior. Trends in Neuroscience and Education, 2, 48–55. Fazio, L. K., Bailey, D. H., Thompson, C. A., & Siegler, R. S. (2014). Relations of different types of numerical magnitude representations to each other and to mathematics achievement. Journal of Experimental Child Psychology, 123, 53–72. Feigenson, L., Dehaene, S., & Spelke, E. (2004). Core systems of number. Trends in Cognitive Sciences, 8, 307–314. Halberda, J., & Feigenson, L. (2008). Developmental change in the acuity of the ‘number sense’: The approximate number system in 3-, 4-, 5-, and 6-year-olds and adults. Developmental Psychology, 44, 1457–1465. Lefevre, J-O., Wells, E., & Sowinski, C. (2016). Individual differences in basic arithmetical processes in children and adults. En R. Cohen KAdosh y A. Dowker (Eds.), The Oxford handbook of numerical cognition ((pp. 895-914). Oxford: Oxford University Press. Leibovich, T., & Ansari, D. (2016). The symbol-grounding problem in numerical cognition: A review of theory, evidence, and outstanding questions. Canadian Journal of Experimental Psychology, 70, 12-23. Libertus, M. E., Feigenson, L., & Halberda, J. (2011). Preschool acuity of the approximate number system correlates with school math ability. Developmental Science, 14, 1292-1300. Lipton, J. S., & Spelke, E. S. (2005). Preschool children’s mapping of number words to nonsymbolic numerosities. Child Development, 76, 978-988.
144 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
Linsen, S., Verschaffel, L., Reynvoet, B., & De Smedt, B. (2014). The association between Children’s numerical magnitude processing and mental multi-digit subtraction. Acta Psychologica, 145, 75-83. Lyons, I. M., & Ansari, D. (2015). Foundations of Children’s Numerical and Mathematical Skills: The Roles of Symbolic and Nonsymbolic Representations of Numerical Magnitude. En J. B. Benson (Ed.), Advances in Child Development and Behavior, Vol. 48, (pp. 93-116). Burlington: Academic Press. Lyons, I. M., Price, G. R., Vaessen, A., Blomert, L., & Ansari, D. (2014). Numerical predictors of arithmetic success in grades 1-6. Developmental Science, 17, 714-726. Melby-Lervåg, M., Lyster, S. H., & Hulme, C. (2012) Phonological skills and their role in learning to read: A meta-analytic review. Psychological Bulletin, 138, 322–352. Mundy, E., & Gilmore, C. K. (2009). Children’s mapping between symbolic and nonsymbolic representations of number. Journal of Experimental Child Psychology, 103, 490–502. Nosworthy, N., Bugden, S., Archibald, Evans, B. & Ansari, D. (2013). A two- minute paper-and-pencil test of symbolic and nonsymbolic numerical magnitude processing explains variability in primary school children’s arithmetic competence. PLoS ONE, 8(7), e67918. Orrantia, J., San Romualdo, S., Matilla, L., Sánchez, R., Múñez, D. & Verschaffel, L. (2017). Marcadores nucleares de la competencia aritmética en preescolares. Psychology, Society, & Education, 9, 121- 124. Reynvoet, B., & Sasanguie, D. (2016). The symbol grounding problem revisited: a thorough evaluation of the ANS mapping account and the proposal of an alternative account based on symbol-symbol associations. Frontiers in Psychology, 7:1581. Riley, N. S., Greeno, J., & Heller, J. I. (1983). Development of children’s problem solving ability in arithmetic. In H. P. Ginsburg (Ed.), The development of mathematical thinking (pp. 153-196). New York: Academic Press. Sasanguie, D., Göbel, S. M., Moll, K., Smets, K., & Reynvoet, B. (2013). Approximate number sense, symbolic number processing, or number- space mappings: What underlies mathematics achievement? Journal of Experimental Child Psychology, 114, 418-431.
Revista de Educación, 381. July-September 2018, pp. 127-146 145 Received: 04-07-2017 Accepted: 24-02-2018 Orrantia, J., San Romualdo, S., Sánchez, R., Matilla, L., Muñez, D., Verschaffel , L. Numerical magnitude processing and mathematics achievement
Schneider, M., Beeres, K., Coban, L., Merz, S., Susan Schmidt, S., Stricker, J., & De Smedt, B. (2017). Associations of non-symbolic and symbolic numerical magnitude processing with mathematical competence: a meta-analysis. Developmental Science, 20: n/a, e12372. doi:10.1111/ desc.12372. Siegler, R. S. (2016). Magnitude knowledge: the common core of numerical development. Developmental Science, 19, 341-361. Vanbinst, K., Ghesquière, P., & De Smedt, B. (2015). Does numerical processing uniquely predict first graders’ future development of single-digit arithmetic? Learning and Individual Differences, 37, 153- 160. Xenidou-Dervou, I., Molenaar, D., Ansari, D., van der Schoot, M., & van Lieshout, E. C. D. M. (2017). Nonsymbolic and symbolic magnitude comparison skills as longitudinal predictors of mathematical achievement. Learning and Instruction, 50, 1-13. Yuste, C., Franco, J., & Palacios, J. M. (2013). Test ICCE de Inteligencia. Madrid: ICCE Publicaciones. Yuste, C., Yuste, D., Martínez, R. & Galve, J. L. (2012). BADyG. Batería de Aptitudes Generales y Diferenciales. Madrid: CEPE.
Contact address: Josetxu Orrantia. Universidad de Salamanca. Facultad de Educación. Departamento de Psicología Evolutiva y de la Educación. Pº de Canalejas, 169. 37008 Salamanca. E-mail: orrantia @usal.es
146 Revista de Educación, 381. July-September 2018, pp. 127-146 Received: 04-07-2017 Accepted: 24-02-2018 Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
Analfabetismo en el primer tercio del siglo XX: la creación de escuelas en Valdés (Asturias)
DOI: 10.4438/1988-592X-RE-2017-381-384
Patricia Suárez Trinidad Suárez Universidad de Oviedo
Abstract: This article analyzes the creation of schools in the municipality of Valdés (Asturias) during the first third of the 20th century. It is worth noting the protagonist role played by residents in the materialization of access to education for their children. The residents themselves incurred part of the construction costs of the buildings and even donated the lots where they would be raised and the materials used. The period covered by the Second Republic was the most fruitful in terms of erecting school buildings, at the hands of its mayor Luis Ochoa de Albornoz, who put great effort into meeting the demands of the residents channeled through the Neighborhood Leaders and Watchmen.
Keywords: illiteracy, schools, Asturias, Second Republic, citizens.
Resumen: En este artículo se analiza la creación de escuelas en el municipio de Valdés (Asturias) durante el primer tercio del siglo XX. Cabe señalar el papel protagonista que tuvieron los vecinos en la materialización del acceso a la enseñanza de sus hijos. Fueron los propios vecinos quienes sufragaron parte de los gastos de construcción de los edificios, e incluso donaron los solares donde se habrían de levantar los mismos, así como los materiales utilizados. El periodo que abarca la
Revista de Educación, 381. July-September 2018, pp. 147-171 147 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
Segunda República fue el más fructífero en cuanto a levantamiento de edificios escolares, de la mano de su alcalde Luis Ochoa de Albornoz, que puso gran empeño en atender las demandas de los vecinos canalizadas a través de los Celadores y Alcaldes de Barrio.
Palabras clave: analfabetismo, escuelas, Asturias, Segunda República, ciudadanos.
Introduction
The main objective of the work is to link the process of the institutionalization of education with the advent of the Second Republic in Spain. Understanding this historical evolution is essential because in this context, the issue of education came to occupy a preponderant place that materialized in the creation of schools in remote and inaccessible geographical environments. If something can be affirmed with confidence, it is that the Spanish rural population were the “great forgotten”, despite being the most numerous until well into the twentieth century. And perhaps the educational vacuum was the most burdensome of all they had to endure and caused most of their ills because the generations succeeded without the smallest glimpse of education. According to Bartolomé Cossío, perhaps the most eminent Spanish pedagogue in the late nineteenth and early twentieth centuries, it is fundamental to “teach the child to see from the first day” (Bartolomé Cossío, 1985, p. 49), hence the importance of teachers and schools. The weaker the education of parents, the greater the sense of the development of a network of schools to fill that gap and stimulate the development of these children and of generations to come. Schools as such hardly existed, physically occupying the worst places. Using almost the exact words of Fernández Soria and Agulló Díaz (2004), schools were located in dilapidated homes, in basements or in old stables. As if that were not enough, rural schools were single classroom schools most of the time because the enrollment and the means were insufficient to form a school with grades, which were reserved for the urban environment. This fact meant a handicap for the teacher, who dealt with a large number of students of different ages in the same space, with
148 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias) all that this entailed. As might be expected, the furniture and teaching material went hand in hand with the panorama of prevailing misery. Because of all the above, a high mobility of teachers occurred from the rural area. As soon as they could, these teachers headed to cities or at least to larger populations, where there was a grade school that fulfilled their expectations and offered them greater comfort. It is worth asking at what point the family is permanently displaced by a school in regard to providing the children with a general education that guarantees their literacy, that is, the control of the elementary techniques of reading, writing and counting. Economists consider that education is an investment that allows individuals to improve their future capacity to produce and generate a greater profit and, therefore, will positively affect their well-being. In this sense, parents expressed their wish for the prosperity and greatness of their country through a greater culture for their children in the multiple requests they made for the erection of schools at the beginning of the 20th century (see appendix). The empirical core of this work focuses on Valdés, a municipality of the Principality of Asturias. In this region, in the late nineteenth and early twentieth centuries, the mastery of the three rules (reading, writing and counting) was acquired mainly in the family environment (among those who could afford it) or in one of the few existing schools. In this context, the contribution of emigrants to America was decisive in promoting the construction of schools and eradicating illiteracy1. The structure of the article is as follows. First, we analyze the historical background of the most important educational laws, which allows us to understand the situation regarding the scarcity of schools in the first part of the 20th century. In the second section, the funds and files available for the study of the schools that were built in the first third of the 20th century, namely, information from the Historical Archive of Asturias and the Municipal Archive of Valdés, are analyzed. The empirical study focuses on the origins of the schools of Valdés, most of them raised by Mayor Luis Ochoa de Albornoz, who gave course to the demands of the residents of the council, who wanted literacy for their children. Finally, this work ends with a section of conclusions.
(1) A detailed analysis of this issue can be found in the report prepared by the Inspector of Early Education in Oviedo, Mr. Benito Castrillo Sagredo (1926), available in the Virtual Library of the Principality of Asturias.
Revista de Educación, 381. July-September 2018, pp. 147-171 149 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
1. Brief historical background
Due to being an important aspect in the configuration of the reality that we are considering, it is pertinent to review the legislation gestated throughout the historical period that we aim to approach, taking into account that a legislation’s mere publication often did not mean its fulfillment, creating without a doubt a gap between what was projected and the reality. The first steps in the regulation of public schools in Spain occurred in the time of Charles III with the promulgation of the Royal Provision of July 11, 1771, initiating a timid piece of legislation regarding the appropriateness of education for boys and girls, the requirements to practice as a teacher, school books, and so on. The next milestone would be the Constitution of 1812, whose title IX, article 366, states the following:
“In all the towns of the monarchy, early schooling (primeras letras) will be established, in which the children will be taught to read, write and count and the catechism of the Catholic religion...”
Tortella and Núñez (2011) emphasize the importance of the Quintana Report, which reflected the constitutional precepts of 1812 and the ideas of renowned enlightened people, such as Jovellanos. Manuel José Quintana presided over the commission that issued the mentioned Report in 1813, designating the State as responsible for education and establishing the principle of generality and gratuity. In spite of this, it was not until 1857 that the first General Law of Public Instruction in Spain, known as the Moyano Law, was published, as this was the minister in charge of propelling it and who took as reference the aforementioned Quintana Report. The Moyano Law established three levels that even today, with certain modifications, last: elementary, baccalaureate, and higher or university education. It establishes the obligatory nature of an elementary education that would go from the age of six to nine, taken in schools segregated by sexes that would be located in populations of more than 500. The law states that financial assistance would be only for those who did not have the means to pay, which in practice resulted in the marginalization and labeling of poor children. In addition, the principle of generality was
150 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias) limited to the few existing schools. The operation of elementary schools was left to the local boards of primary education and, by virtue of the Concordat of 1851, the Catholic Church was allowed to intervene actively in education, especially having the right to co-responsibility in secondary education. The configuration of educational institutions of a public nature began with higher education, which served as a spearhead for the development of elementary schools throughout the nineteenth century, linked to the action of the State as a means to reinforce their unification (Egido Gálvez, 1994). This fact is also underlined by Núñez (2005), who emphasizes the importance of the creation of the Ministry of Public Instruction and Fine Arts in 1900 as the peak of the implementation of primary education. In the decade of the twenties of the last century, the situation was alarming: Sixty years after the enactment of the Moyano Law, the reality was that there were hardly any schools for half the population of school age. López Martín (1997) states that the increase in the creation of schools in the first third of the twentieth century did not eliminate the problems of illiteracy in Spain and that the network of schools was still meager and incidental. As an example, in 1900, the percentage of illiterates in Spain was approximately 65% of the population, and in 1920, it was 52% 2. Thus, the Primo de Rivera regime marked as one of its objectives in educational matters the construction of schools, although without forgetting its role as a vehicle of the expansion of the ideological values of the dictatorship. Although there is controversy as to the total number of schools erected in the period from 1923 to 1930, it seems that they were not less than 6,000, the effort being praiseworthy even if the deficit situation continued to be maintained. López Martín (1994) analyzes in detail education in the dictatorship of Primo de Rivera and notes that this constructive effort was favored by the relative economic bonanza of the twenties, a result mostly of the neutrality of Spain during the Great War. As an example, according to the Statistical Yearbook of Spain cited in Lopez Martin (1994), in the province of Oviedo, it was estimated in 1920 that there was a total of 118,920 inhabitants of school age, for a total of 1,327 schools in 1923, with a school population similar to provinces such as Madrid, Barcelona and Valencia, forming a quartet that exceeds the 100,000 barrier.
(2) See López Martín (1994).
Revista de Educación, 381. July-September 2018, pp. 147-171 151 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
In 1923, Asturias was the Spanish province that had a largest number of schools in absolute terms, reaching a ratio of almost 90 students per center. This fact is mainly due to its particular geographic and population configuration and to the valuable economic contribution of emigrants to America. Asturias was a pioneer in the industrialization of the country thanks to the coal basins of Mieres and Langreo, which generated an increase in educational demand without precedent. With the advent of the Second Republic, education came to occupy a preponderant place, promoting the creation of numerous schools because it was estimated that more than one million children were out of school. The words of the Professor of Contemporary History of the University of Salamanca Francisco de Luis are eloquent, “education is the basis of democracy that the Republic wants to establish” 3. A five-year School Construction Plan never before observed was launched, reaching approximately 16,000 of the 27,000 planned. Hernández Díaz (2000), in his work on rural schools, notes the scarcity of the resources of the municipalities, especially in post-war Spain, and the necessary direct provision of peasants in the construction of new schools. In addition, with the arrival of Franco in power, once the regime was consolidated, the Law of Primary Education was published in 1945, in which the Church played a preponderant role, collecting in it “the Christian principle of the separation of the sexes in teaching”. The Law established four periods of school: 1) the period of initiation, which included nursery schools for children up to four years and nursery schools for children from four to six years old, 2) the period of elementary education, for children from six to ten years old, and 3) the perfecting period, for children from ten to twelve years old. These latter two stages were strictly obligatory (six to twelve years). Lastly was 4) the period of professional initiation, for children from twelve to fifteen years old. Puelles Benítez (1991) also highlights the confessional nature of the teaching as well as the sloppiness of the dictatorship in promoting the creation of new schools despite the existing deficit.
(3) See documentary titled “The Republic of teachers” https://www.youtube.com/watch?v=JVjommbxQSU
152 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
2. The schools in Asturias
The Asturian school radiography of the first half of the nineteenth century was based on early education schools (primeras letras) inherited from the old regime, rooted in the neighborhood community and economically dependent on varied resources, some with associated properties, others supported by the neighborhood boards, dependent on Foundations or of the Church (Mato Díaz, 2010). In rural Asturias (note that at that time the majority of the population settled in rural areas in a very dispersed way), it was normal for schools to be supported by neighborhood boards and taught by unqualified teachers, and in the case of small and isolated villages, the instruction focused on a small period of the year that coincided with the time when no important work was carried out in the field, giving rise to the temporary schools, which represented a quarter of the total schools, according to Mato Díaz (1992). It would be the Moyano Law of 1857 that determined the control of education by the State, establishing the endowment of a school for boys and one for girls for each five hundred inhabitants. The law also designated the municipalities as responsible for the maintenance of schools. However, in Asturias, a rigorous application of the law would require the implementation of double the existing schools, so through the special arrangement of February 21, 1860, the municipalities were exempted from such responsibility, thereby perpetuating the inherited situation, which meant the deschooling of a large sector of the child population. Regarding the training of teachers, although beginning 1844, there was a Teacher School (Escuela Normal de Maestros) in Oviedo, the graduates were few and did not cover all the existing vacancies, only those of the Higher Primary Schools and the Complete Schools, leaving the rest in the hands of qualified teachers at best and teachers with no diplomas, known as “half-year teachers.” Luis Bello states in his work Viaje por las escuelas de España (1926) that these “teachers” hired by the residents of towns that did not have school ate every day in a house, hence the denomination of catapotes and babianos (from Babia and Omaña), northwest regions of León bordering with Asturias, who were appreciated because they spoke Castilian correctly. In contrast, children from very rural areas used to have as a mother tongue different forms
Revista de Educación, 381. July-September 2018, pp. 147-171 153 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
of bable (a regional dialect) 4. Indeed, Soler Mata (2008, p.15) notes the need for teachers with degrees to occupy rural schools in such a way that the image of the school is the “face of the teacher”. In Gera (Tineo-Asturias), during All Saints, a famous cattle festival, the neighbors of the towns took advantage of contracting teachers, with the term “half year teachers” arising because there was school from November to May 1, leaving the summer season for the work of the field, in which child labor was essential. Following the legislative review of this subject, the rest of the section aims to reflect the situation of education in a particular municipality, Valdés. The section is structured as follows: it explains the origin of the information collected as well as the usefulness of the files for educational historiography. Next, the documentation gathered with the objective of clarifying the origin of the Valdesan schools is explored in detail. Finally, a reference is made to the early education schools of the municipality of Valdés.
2.1. Funds for educational inspection and statistics of primary education
The main source used in this article were the funds of the Provincial Archives of Asturias, in which there is information concerning teaching collected by the Inspection Service of the Ministry of Education 5. The National Statistics Institute prepared national statistics called “Primary Education Statistics” that collected information about the center, teachers, and students, as well as adult education. As for the data of the center, information was collected on the regime of the center, whether it was state owned, within state centers if it was funded by ordinary provision or received special funds (Patronato) and non-state centers if it depended on the Church or was run privately. Regarding the class of the center, we distinguish between
• Single-classroom schools, which were those in which a single teacher was in charge of teaching all the different courses, according to the period of compulsory schooling. These schools could be for
(4) See Bello (1926). (5) Information from the school identification cards 1958-1965. Box 11292.
154 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
boys, for girls or mixed, the latter occurring in areas where there was not enough enrollment to provide two schools. • Multi-classroom schools, which were educational entities formed by a variable number of units (never more than eight) whose teachers were in charge of teaching the different compulsory primary school courses. • School groups, which were primary educational entities constituted by eight or more school units, for students of the same sex, with a “principle without a course”.
Regarding the data of the teaching staff, information was collected on whether there were principals without a grade and the number of teachers, divided by sex. In relation to the data of the students, a template was used that showed the number of students according to the age, the average attendance broken down by sexes, the number of certificates issued in the previous year, primary studies or schooling, the number of internal students, and half-pension (who offer at least one meal facilitated by the center itself) and free centers (no fee or pension for the normal class day). Finally, information was collected on adult education, that is, on the number of students enrolled, the average attendance, and those who did not know how to read or write at the beginning of the course. For example, in Oviedo, according to a report prepared by the Chief Inspector in 1970, 3,015 children without schooling or very low literacy were counted, out of a total school-age population of 18,453 . These figures are of note because it is precisely a municipality like Oviedo, the capital of the province, that is well communicated if we compare it with other areas of the region. In addition, it occurred on such a recent date, which leads us to think about the situations undoubtedly experienced in more distant dates and in less urbanized and more remote and inaccessible places of Asturias.
2.2. The schools in the council of Valdés
At the end of the sixties, according to the Primary Education Statistics mentioned in the previous section, the number of schools in the Valdés council was 95, of which 31 were mixed schools, 28 were single-
Revista de Educación, 381. July-September 2018, pp. 147-171 155 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
classroom girls’ schools, 28 were single-classroom boys’ schools, and 6 were kindergartens. In addition, a multi-classroom for girls and another for boys were both located in Luarca, the capital of the council. Whereas this section is devoted exclusively to analyzing the distribution of the schools that existed at the end of the sixties in the council of Valdés by town and parish, the following section analyzes the historical origins of some of those schools through the documentation found in the Valdés City Council funds. The occupational structure of the municipality’s population in 1960 showed the characteristics of a still-agricultural society, with 70% of its active population dedicated to agricultural activity. The weight of this activity within the rural municipalities was 54%, so despite Luarca being a municipality of more than 10,000 inhabitants, it is classified as an intermediate urban area, and its economic structure was mainly rural. By way of summary, Table 1 shows the distribution of the active population by activity groups in the province, Oviedo and Luarca.
TABLE 1. Active population by economic activity groups.
Provin- Provin- Oviedo Luarca Oviedo Luarca ce ce % % % TOTAL Active population 391.117 40.621 9.572 - - - Agriculture, forestry, hunting 151.665 1.739 6.767 38,8 4,3 70,7 and fishing Exploitation of mines and 54.854 1.388 25 14,0 3,4 0,3 quarries Manufacturing industries 65.463 8.947 764 16,7 22,0 8,0 Building 23.428 4.726 615 6,0 11,6 6,4 Electricity, water, gas and 2.670 609 33 0,7 1,5 0,3 Sanitary services Commerce 22.905 5.242 336 5,9 12,9 3,5 Transportation, storage 17.136 2.991 177 4,4 7,4 1,8 and communications Public services and 40.575 13.508 533 10,4 33,3 5,6 services with people Unspecified activities 12.421 1.471 322 3,2 3,6 3,4
Source: Own elaboration based on data from the 1960 Census (National Institute of Statistics).
156 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
In view of the data offered in Table 1, it seems reasonable to assume that the occupational structure in the first half of the 20th century was eminently rural. The low rate of urbanization coupled with very poor infrastructure justified the provision of schools in the main towns of the council. Most of the children went to school by walking several kilometers, so the absence of a school in a radius of more than six kilometers impeded the schooling of the population. The families lived at the foot of the land they cultivated and where they were given food, at best in their own farmhouse or in the houses where they served as servants or under a sharecropping regime. The children were very necessary labor, especially at certain times of the year, namely the planting and harvest periods. Table 2 shows the distribution of these schools in the 59 towns of the council. In general terms, the predominant type of school is mixed, with 31 of the 59 towns having schools where this type of center exists, whereas in 28 of the 59 villages, there are single-classroom schools for boys and girls. The existence of single-classroom schools for boys and girls compared to mixed schools seems to be due to a demographic criterio. The nuclei with mixed schools are in principle smaller, and therefore, the number of children of school age is lower; this is the case of Alienes, Argumoso, Artosa, Biescas, Brañarronda and many others. These towns are currently very unpopulated, and the inhabitants who remain are usually of advanced age. In less than a century, the reality has turned totally opposite. Whereas at the beginning of the XX century the lifting of school buildings was peremptory, at the end of the same century the majority of the rural schools had closed for lack of registration due to the brutal population exodus suffered in this area.
Revista de Educación, 381. July-September 2018, pp. 147-171 157 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
TABLE 2. Statistics of schools in Valdés, 1958-1963
Multiple Multiple Single Single class- class- Kindergar- class- class- Mixed Total room of room tens room of room girls boys girls boys Aguera-Paredes 1 1 2 Alienes 1 1 Almuña 1 1 2 Arcallana 1 1 2 Argumoso 1 1 Artosa 1 1 Ayones 1 1 2 Barcia 1 1 1 3 Biescas 1 1 Brañarronda 1 1 Brieves 1 1 2 Buseco 1 1 Busto 1 1 2 Cadavedo 1 1 1 3 Cadollo 1 1 Canedo-Otur 1 1 Canero 1 1 1 3 Carboniella 1 1 2 Carcedo 1 1 2 Caroyas 1 1 2 Castañedo 1 1 2 Colinas 1 1 Concernoso 1 1 El Otero 1 1 2 El Vallín 1 1 Ferrera 1 1 La Mafalla 1 1 La Montaña 1 1 2 La Mortera 1 1 2 Lago 1 1 Leiriella 1 1
158 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
Lendepeña 1 1 Luarca 1 1 1 1 1 5 Menudero 1 1 Moanes 1 1 2 Muñás 1 1 2 Otur 1 1 1 3 Paladeperre 1 1 Paredes 1 1 2 Pereda 1 1 Piñeros 1 1 Pontigón 1 1 2 Quintana 1 1 2 Ranón 1 1 Rellón de Merás 1 1 2 San Feliz 1 1 San Pelayo de 1 1 Tahona Santiago 1 1 2 Setienes 1 1 2 Sexmo 1 1 Silvamayor 1 1 Siñeriz 1 1 Tablizo 1 1 Telares 1 1 Trevías 1 1 1 3 Valtravieso 1 1 Villademoros 1 1 Villagermonde 1 1 Villanueva- 1 1 2 Cortina Total 1 1 31 6 28 28 95
Source: Own elaboration based on the Primary Education Statistics of the National Institute of Statistics. Box 11292 of the Historical Archive of Asturias.
Analyzing the number of schools per parish, in Canero, La Montaña and Santiago, the number is ten or more. In all the towns that were parish headed and, therefore, the most populated nucleus of the parish,
Revista de Educación, 381. July-September 2018, pp. 147-171 159 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
there were single classroom schools for boys and girls. This is a rule except in the parish of Alienes, with a small extension and very low population density. While section 2.4 deepens the knowledge of kindergarten schools, the six kindergarten units were located in the most populated parishes, with significant economic activity located on the coastal plain, except for Trevías, which is located more inside the council. Logically, the degree of “modernization” of these places is far superior to that of the most distant parishes of the capital, fully immersed in an agrarian economy close to self-sufficiency in many ways.
TABLE 3. Statistics of schools by parish in the council of Luarca, 1958-1963
Multiple Multiple Single Single Kindergar- classroom classroom Mixed classroom classroom Total tens of girls boys of girls boys Alienes 2 2 Arcallana 5 1 1 7 Ayones 1 1 1 3 Barcia 2 1 1 1 5 Cadavedo 1 1 1 1 4 Canero 1 1 4 4 10 Carcedo 2 2 4 Castañedo 1 1 1 3 La Montaña 8 2 2 12 Luarca 1 1 1 2 2 7 Muñás 2 2 2 6 Otur 1 1 1 1 4 Paredes 1 3 3 7 Santiago 2 4 4 10 Trevías 4 1 3 3 Total 1 1 31 6 28 28 95
Source: Own elaboration based on the Primary Education Statistics prepared by the National Institute of Statistics. Box 11292 of the Historical Archive of Asturias.
The council of Valdés stands out for having the largest number of pasture lands (brañas) in all of Asturias. These are small towns located
160 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias) in high areas with steep slopes. Eleven of these pasture lands had their own school: Artosa, Argumoso, El Vallín, Cadollo, Buseco, Brañaronda, Siñeriz, Silvamayor, Lendepeña, Leiriella and Lago. The architectural typology is diverse insofar as it responds to different realities, times and economic means. Thus, we see that the best buildings are concentrated in Luarca and in wealthy towns such as Arcallana, which has a manor house of two floors. Regarding the construction of a large part of the single classroom schools built in the first third of the twentieth century, it should be noted that they present practically identical designs, composed of a central body, which normally housed the library, and two wings, one for boys and another girls, equipped with large windows on both sides (three or four depending on the size of the same) giving a design that provided plenty of light. This is the case of the schools of Muñás, Carcedo, Ferrera, Brieves, Villanueva, Silvamayor, Barcia, Cadavedo, Otur and Paredes. Only in Luarca was there a multi-class school for girls and another for boys, the Mixed School Group of Campo de la Feria, “with its sections in two or more buildings” according to the Statistics of Primary Education of 1965.
Photo of the Muñás School (Archive by Eduardo Domínguez Sánchez-Pico)
Revista de Educación, 381. July-September 2018, pp. 147-171 161 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
Photo of the School of Arcallana (Author Trinidad Suárez Cano)
2.3. On the origin of valdesian schools
This section deepens the understanding of the birth of some of the schools of the council, taking as a main thread information from the review of the documentary archives available in the City of Valdés. In general terms, it is worth noting the contribution of migrants to building schools (Lombardero Rico, 2013). Both those most favored by fortune and the more modest participated, sending what they could through collections. A large number of schools were raised with the money of these emigrants, especially before the advent of the Second Republic, at which time the State became aware of the importance of education and established an unprecedented school construction plan. As an example, we record the work of Ramón Asenjo, who emigrated to Argentina and upon his return became mayor of Luarca for several years. Ramón Asenjo donated money personally and also promoted from City Hall the construction of several schools (Arcallana, Ayones, Barcia, Cadavedo, Luarca, Muñás, Canero, Paredes, Santiago, Trevías, La
162 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
Carboniella, Carcedo and Otur, among others). According to the Report of First Teaching of D. Benito Castrillo Sagredo (1926),
In Ayones, D. Vicente Trelles gave 700 pesetas, D. Ramón Asenjo for several schools a respectable amount; in Canero, D. Manuel Arango gave the lot and money in total 4,000 pesetas, the emigrants also gave 1,687 pesetas for the Library, and 1000 for a typewriter; in Carcedo 13,000 pesetas were received from Cuba; in Luarca, part of the material and a sewing machine are donated by the latter by D. Francisco Gamoneda; in Muñás, the Americans gave 9,000 pesetas, in Otur the Most Excellent D. Ángel García, Viceconsul of Spain in Rosario de Santa Fe and his brother D. Ramón contributed 12,000; to Paredes came from America 10,000, the site was donated by Mrs. Germana González; in Santiago the plot was donated by the popular banker, D. Vicente Trelles and Mr. Marqués de Ferrera; in Trevías popular subscription produced 19,000 pesetas, the State gave 5,000: it has another donation for typewriter and circulating library; in La Carboniella subscriptions reached 7,500.
In the aforementioned report by D. Benito Castrillo Sagredo (1926, pp. 52-56), it is stated: “in other schools built there, the State did not intervene”. Of note is the creation of the school of Rellón de Merás because it was built in 1913 by Manuel García Merás Sánchez, an immigrant to Cuba, who also had two attached houses for teachers. A similar case was the school of Lendepeña, inaugurated in 1927, built at the expense of Nicolás Gayo, a resident of the town who also emigrated to Cuba. In addition to building the school building, he also constructed the road linking Lendepeña to Arcallana, the head of the parish, and the chapel. The testimony of Valentín García Nido, born in Lendepeña in 1928, is quite eloquent:
I started going to the village school at 8 years old. I had to walk about two kilometers on a very steep climb to get to school, and the boys and girls who came from the towns of Arquillina, Cerezal (Salas council) and Los Baos traveled several kilometers every day to go to school of my town, which became arduous when the cold and the rain increased. Half the days they did not come.
Revista de Educación, 381. July-September 2018, pp. 147-171 163 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
Diving into the origin of the multi-classroom school of Luarca, one of the oldest in the region, we find that in 1906, the City Council auctioned the project for the construction of public schools previously in the “Vega del Campo “, whose construction was carried out with the help of emigrants. In 1910, they ceased being municipal schools and became multi-classroom schools, which is how they will be known in the future, through a Royal Decree. After the Civil War, they took the names of “José Antonio” and, currently, “Father Galo”. In the rural area, it is necessary to point to the little-studied fact of the protagonist role played by the residents in the materialization of access to the education of children, which until then was very deficient in all aspects. It was the residents themselves who paid part of the construction costs of the buildings, donated the lots where they would be raised, and asked the councils to hire a teacher, whose accommodations also used to be borne by the neighborhood. Among the cases found, it is worth noting the petition filed by the residents of the towns of Ore, Pontigón, Llaneces, Landedo and Bustiello to the Mayor-President of the City Council and Local Board of Early Education of Luarca in the proposal for a Mixed National School in Pontigón (see in the appendix the transcript of the proposal dated in 1927). In addition, this document reflects the fact that the Unified Schools of Carcedo were previously created following the same procedure that is set forth below, and everything seems to indicate that it was the usual process in the construction of most of the council’s schools. The residents of the town of Piñeros de La Montaña maintained a school beginning at least in 1905, with the help of a small subsidy from the municipality consisting of 150 pesetas per year; the residents built the neighborhood exclusively on their own. In the year 1931, in view of the agreements of the Ministry of Public Instruction to erect a considerable number of schools, the creation of a mixed school group to which the children of Siñeriz and Degollado, towns that did not have a school and were a short distance, could also go was requested. Once the Piñeros school was created, the residents of Siñeriz also mobilized to create a school in their town. The neighbors of the towns of La Mortera, Mones, Espiniella Alta, Espiniella Baja and Candanosa, which form the school district of La Mortera, in 1932, seeing that the auction of the buildings of the school buildings of Brieves, Silvamayor, Villagermonde, Siñeriz, Otero, Ferrera
164 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias) and Villanueva had been announced, asked the municipal corporation to include the construction of a school in La Mortera, offering the land for the site and a cash amount of 3,480 pesetas. In Tablizo in 1933, whose school district encompasses Tablizo, Gallinero and Busmarzo, the school operates in a privately owned building, leased by the City Council, while another more adequate building was built. In the municipality of Valdés, some 32 schools were built between April 28, 1931, and October 5, 1934, during which time Luis Ochoa de Albornoz was mayor (see the transcript dated June 8, 1932), which is equivalent to saying that more than a third of the schools in the council were raised in this brief interval of time.
2.4. Kindergarten schools in valdés
The charitable effort led to the birth of schools to collect young children, from disadvantaged social strata, and driven by private initiative (Molero Pintado, 1999). The industrial revolution was the first major transfer of the population from rural to urban areas and the incorporation of women into the labor market. It is in this context that the first kindergarten schools arose, although still with a marked welfare character (Colmenar Orzaes, 1991). As stated in the previous section, Valdés had six kindergarten schools, Luarca, Barcia, Otur, Canero, Cadavedo and Trevías, although according to a document found in the Municipal Archive of Valdés, in 1932, another kindergarten school was opened in the town of Luarca. The one counted in the statistics of the sixties was located in the grounds of the fair. These towns that had kindergartens were the closest to the town of Luarca, with the exception of Trevías, which is located inside the council. If we were to ask ourselves the reasons for its origin, as well as the number of inhabitants and the attraction pole that Luarca assumed as the capital of the council, we would see that these populations had non- strictly agricultural sector economic activity, as was the case in most Valdesian nuclei, eminently agricultural and livestock6. In Luarca, the school of kindergarten of the manor (escuela de párvulos del casco) was opened in 1932, created by Order of November
(6) For a more detailed analysis of kindergardens in Valdés, see Suárez and Suárez (2015).
Revista de Educación, 381. July-September 2018, pp. 147-171 165 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
26, 1932 (Gazette of December 1), with the building of private property, at which the school of Arts and Vocations was previously located, leased by the City Council, and at which pertinent reforms were carried out. It was launched on a provisional basis while a more adequate one was being built. Its creation responds to the growing demand for enrollment for the youngest children. According to the documentation handled, breakfast and lunch were provided to the students because there was a kitchen for that purpose. This last fact makes us think that although the majority of the towns did not have proper childhood education set up in a kindergarten school and such education was not compulsory, in practice they were accepted by teachers if they were willing to learn and if their parents considered it. The Barcia kindergarten school was created in 1933 as part of the project to build school buildings in all the villages of the council where a school was set up, in the words of the Republican mayor Luis Ochoa de Albornoz.
Conclusions
It was not until the twenties of the last century that a school network with properly qualified teachers and decent and hygienic facilities was put into operation in the council. Before then, schools were placed where feasible, in granaries, private houses or barns, and teachers themselves took care of it while lacking the proper preparation. Thus, it can be deduced that the authorities up to that moment did not have as a priority the establishment and consolidation of primary education and that they had to be spurred in a certain way by the interested parties themselves to leave their laziness behind. The main conclusion is that it is not the State but the residents of towns that demand help to open schools; they contribute by donating the land, through considerable economic contributions given their precarious economic situations and their own work in the construction and hauling of materials donated by themselves, mainly stone and wood. What is observed after the analysis of the information collected is that the parents, eager for their children to have a better life than theirs and for the eradication of illiteracy in their town, demanded that the City Council of Luarca create schools all throughout the council, making the period
166 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias) covered by the Second Republic the most fruitful in terms of erecting school buildings, at the hands of its mayor Luis Ochoa de Albornoz, who put great effort into meeting the demands of the residents. Another aspect to highlight is the enormous work carried out by the emigrants, being exclusive promoters of several schools, and through economic contributions that came from Cuba, Argentina, Mexico, Puerto Rico and Guatemala, contributing to the construction and support of numerous schools. One wonders what would have happened if so many and such generous men had not left their homes on their way to the Americas. In the end, with their patronage, they eradicated illiteracy in their beloved towns thanks to the money that crossed an entire ocean. Certainly, the creation of a school in a town meant grounding its population, improving the cultural level of its inhabitants and providing them with the minimum tools to face the tremendous socio-economic change that Spain experienced in the 1960s. Today, the rural schools of Valdés are already past. The last ones, Muñás and Concernoso, closed their doors in 2016 and 2017, respectively. All of them have become part of that nostalgic place where oblivion lives.
Bibliographic References
Bartolomé Cossío, M. (1985). Una antología pedagógica. Madrid: Ministerio de Educación. Bello, L. (1926). Viaje por las escuelas de España. Madrid: El Magisterio Español Colmenar Orzaes, C. (1991). Las escuelas de párvulos en España durante el siglo XIX: su desarrollo en la época de la Restauración. Historia de la Educación, vol. 10, 86-110. Egido Gálvez, I. (1994). La evolución de la enseñanza primaria en España: organización de la etapa y programas de estudio. Tendencias pedagógicas, nº 1, 75-86. Fernández Soria, J. M. y Agulló Díaz, M. C. (2004). Una escuela rural republicana. Valencia: Universitat de Valencia. Hernández Díaz, J. M. (2000). La escuela rural en la España del siglo XX. Revista de Educación, núm. extraordinario, 113-136.
Revista de Educación, 381. July-September 2018, pp. 147-171 167 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
Lombardero Rico, Ch. (2013). La huella de la emigración a América en el cantábrico occidental. Gijón: CICEES. López Martín, R. (1997). La construcción y creación de escuelas en la España del primer tercio del siglo XX. Historia de la educación, nº 16, 65-90. López Martín, R. (1994). Ideología y educación en la dictadura de Primo de Rivera. Escuelas y maestros. Valencia: Universitat de Valencia. Martínez Sánchez, J.J. (2006). La legislación escolar en la II República Española. Un referente de progreso y libertad en la escuela pública. Available at: http://martinez-carmona.es/libros/Legislacion_escolar_ en_la_II_Republica_espanola.pdf Mato Díaz, A. (2010). Las escuelas y los maestros de primeras letras (siglo XIX). Magister: revista miscelánea de investigación, nº 23, 19-33. Mato Díaz, A. (1992). La escuela primaria en Asturias (1923-1937). Los procesos de alfabetización y escolarización. Oviedo: Dirección Provincial de Asturias, Ministerio de Educación y Ciencia. Molero Pintado, A. (1999). Bases para una historia de la educación infantil en España. La figura de Eugenio Bartolomé y Mingo. Madrid: Universidad de Alcalá. Núñez, C. E. (2005). Educación. En A. Carreras y X. Tafunell, X. (Ed.). Estadísticas históricas de España Volumen I (155-244). Madrid: Fundación BBVA. Puelles Benítez, M. (1991). Educación e ideología en la España contemporánea. Barcelona: Editorial Labor. Soler Mata, J. (2008). El maestro y la “fisonomía propia” de la escuela rural. Una visión histórica de la escuela rural en Cataluña y España. En N. Llevot Calvet y J. Garreta Bochaca (Eds.) Escuela rural y sociedad (11-41). Universitat de Lleida. Suárez, P., Suárez, T. (2015). Analfabetismo y creación de Escuelas en el Municipio de Valdés (Asturias). Available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_ id=2700043 Tortella, G., Núñez, C.E. (2011). El desarrollo de la España contemporánea. Historia económica de los siglos XIX y XX. Madrid: Alianza Editorial.
Contact address: Patricia Suárez. Universidad de Oviedo, Facultad de Economía y Empresa, Departamento de Economía. Avenida del Cristo s/n. E-mail: [email protected]
168 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
Appendices
1. Transcription from the petition to the Mayor - President of the City Council and Local Board of Early Education of Luarca (Pontigón, March 1 of 1927) Those who write, of legal age, Municipal Wardens of the towns of Ore, Pontigón and Llaneces of the parish of Carcedo, and of those of Landedo and Bustiello of the parish of Ayones, to you and to the corporations of their dignified presidency expose: – That in the aforementioned towns of Ore, Pontigón and Llaneces, with a population census of approximately four hundred inhabitants, there are some sixty-five children of school age who lack the proper instruction because they do not currently have other means of education than a private school subsidized by the City Council, installed in terrible conditions (in a granary) lacking the necessary material and of all pedagogical method, with children therefore having to travel the children distances of 3, 4 and 5 kilometers, astray, to the nearest National School, which is Carcedo’s, in which, on the other hand, there is already excessive school enrollment. – That in the also referred towns of Landedo and Bustiello, with a census of population of one hundred inhabitants and about twenty-six boys and girls of school age, there are no other means of education than the Mixed National School of Ayones, parish to which they belong, about ten kilometers away, also through bad roads, making it impossible during the winter and in the rainy season to travel that distance for the children to attend school. – That in view of the great difficulties listed, of the inescapable duty that parents have to ensure the instruction of their children, thus fulfilling one of our obligations to them and to the Homeland, whose prosperity and greatness, based on a greater culture, we crave; animated by the dignitaries Authorities of the Council and very especially by the Governmental Delegate of this district, in meeting of all the neighbors of these towns, verified in the Pontigón the day twenty-two of August of one thousand nine hundred and twenty-four, a Board was constituted sponsoring the creation of a school group in Pontigón or its surroundings, as it is the central point of all the towns. – That in said meeting, as recorded in the copy of the minutes that we attached, the contributions of all the neighbors were agreed in principle,
Revista de Educación, 381. July-September 2018, pp. 147-171 169 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias)
subscribed cash ascending to the sum of 3,334 pesetas that together with some 1,200 and met by subscription from Cuba, yield a total of 4,534 pesetas, hoping that these subscriptions will increase when it is observed that the realization of the project is already a definite fact. – That in a later meeting also verified in Pontigón at the beginning of the month of last February and summoned to the effect by Mr. Cura, regent of Carcedo D. Inocencio F. Villamil, and by the councilman D. Antonio Castrillón García, to their indications, it was unanimously agreed to ratify the previous offer and expand the commitment of the neighborhood contributions in the following manner: 1º. Cede, free to the City Council, the site for the building indicating the site where it will be located, which has already been resolved with the current owners of the land. 2º. Compromise by mandatory personal provision to carry all the materials necessary for the construction of the work, such as: stone, mud, sand, wood, etc. from the places from which they are extracted and also from the other materials that are imported: cement, lime and slab, from the place where they are unloaded by trucks or other means of transport. Neighbors who do not have cars suitable for that purpose agree to pay the amount of the personal benefit indicated by the respective guard. – And that, both the cash amounts already subscribed and those that could be subscribed in the future, will be delivered as soon as it is necessary, either to the sponsoring Commission or to the City Council according to the Mayor’s Office, following the same procedure when the construction of the Schools of Carcedo, all being forced to the payment of said amounts that will be able to be collected even by the procedure of surcharge if it was necessary. – Therefore, and by virtue of the foregoing we request first the creation of a Mixed National School in Pontigón, if possible managed by a teacher, for which purpose we have, in that town, two rented houses that we believe meet the due conditions: one of which the Priest of Arcallana is empowered and is willing to rent to the City Council for such purpose; and another one from D. Facundo García, a local resident, who spontaneously offers himself, in addition, to provide a house - a free room for the Teacher. – And finally, counting on the contributions of the neighborhood to whose fulfillment is mandatory with the signatures of all of them put forth in separate sheets, that the City Council of its worthy presidency,
170 Revista de Educación, 381. July-September 2018, pp. 147-171 Received: 10-07-2017 Accepted: 23-02-2018 Suárez, P., Suárez, T. Illiteracy in the first third of the twentieth century: the creation of schools in Valdés (Asturias) giving once more proof of its love for teaching, agrees to include in the project of the first constructions of school groups of the council the one of a single classroom school in the mentioned town of Pontigón, parish of Carcedo, so that in Carcedo the boys and girls of the five interested towns can receive instruction. It is justice that we hope to achieve from your honor whose life God keeps many years. Pontigón, March 1st of 1927. El Celador by Ore José Pérez, El Celador by Pontigón Higinio Gómez El Celador de Llaneces Diego Álvarez, El Celador by Landedo Jesús García El Celador de Bustiello José Fernández The Mayor of Barrio Florentino Fernández 2. Transcription of the instance carried out by Mr. Luis Ochoa de Albornoz (Mr. Mayor- President of the City Council) to the Illustrious General Director of Early Education of the province of Oviedo (Luarca, June 8, 1932) Mr. Luis Ochoa de Albornoz, Mayor - President of the City Council of Luarca, accepting the provisions of the Royal Decree of July 10, 1928. To the illustrious we request the grant of EIGHTEEN THOUSAND PESETAS for the direct construction by the Municipality of a new building with destination to two single room national schools, one for boys and one for girls, in the town of Villanueva, this council, to in which are attached the opportune certification of the municipal agreements, the project written by the Architect D. José Ramón del Valle, and the sketch or plan of location for the expressed school building.- Luarca June 8, one thousand nine hundred thirty two.
Revista de Educación, 381. July-September 2018, pp. 147-171 171 Received: 10-07-2017 Accepted: 23-02-2018
Linking coaches and student achievement: PTA in Colombia1
Coaching a docentes y rendimiento académico: PTA en Colombia
DOI: 10.4438/1988-592X-RE-2017-381-385
Ximena Dueñas Herrera Ana Bolena Escobar Escobar Silvana Godoy Mateus Jorge Leonardo Duarte Rodríguez Instituto Colombiano para la Evaluación de la Educación
Abstract The Transformation of Educational Quality Program Todos a Aprender (PTA) is a Situated Professional Development or Coaching program, implemented by the Ministry of National Education since 2012. The program is aimed at the country’s primary education teachers of public schools that have a large quantity of students with difficulties in their learning process. The program’s strategy includes the selection and training of trainers and tutors to support teachers in the development of their classes, and the provision of educational materials to teachers and students. The review of the existing literature shows that the impact of the program on students’ academic performance has not been evaluated so far. Hence, with data from the standardized test Saber 3°, 5° y 9°, applied during four consecutive years by the Colombian Institute for the Evaluation of Education (Icfes) at grade, area and school shift, we try to measure the effect of PTA through the estimation of a linear model and a Logit model with fixed effects on the school performance. The results show that, in most grades and areas, an additional year in PTA correlates positively with student’s performance. Moreover, the effect is more robust in the third grade and math area in schools with a lower socioeconomic level. Although these results must be read with
(1) The authors wish to thank Diana Oquendo Victoria for editing the document.
Revista de Educación, 381. July-September 2018, pp. 173-197 173 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
caution; the results may be useful for decision-making regarding the extension and continuity of the program.
Keywords: Situated Professional Development, Coaching, Tutors, Impact Evaluation, Academic Performance
Resumen El Programa para la Transformación de la Calidad Educativa Todos a Aprender (PTA) es un programa de Desarrollo Profesional Situado o Coaching, implementado en Colombia por el Ministerio de Educación Nacional desde el año 2012. El PTA está dirigido a los profesores de educación Básica Primaria de los colegios oficiales del país que tienen gran cantidad de estudiantes con dificultades en su proceso de aprendizaje. La estrategia del programa incluye la selección y capacitación de formadores y tutores encargados de apoyar a los docentes en el desarrollo de sus clases, y la provisión de materiales educativos para docentes y estudiantes. La revisión de la literatura existente permite concluir que el impacto del programa en el desempeño académico de los estudiantes no ha sido evaluado; por lo cual, se utilizan los resultados obtenidos durante cuatro años consecutivos en la prueba Saber 3°, 5° y 9°, aplicada por el Instituto Colombiano para la Evaluación de la Educación (Icfes) a nivel de grado, área y jornada de los establecimientos, para intentar medir el efecto del PTA mediante la estimación de un modelo lineal y un modelo Logit con efectos fijos a nivel de sede. Los resultados muestran que, en la mayoría de grados y áreas, un año adicional en el PTA se correlaciona positivamente con el desempeño de los estudiantes. En particular, el efecto del programa es más robusto en el grado tercero y el área de matemáticas para las sedes del nivel socioeconómico más bajo. Aunque estos resultados deben leerse con algunas precauciones, proporcionan información que puede ser útil para la toma de decisiones relacionadas con la ampliación y continuidad del programa.
Palabras clave: Desarrollo Profesional Situado, Coaching, Tutores, Evaluación de Impacto, Desempeño Académico.
Introducción
In education, Situated Professional Development (SPD) or coaching is a strategy that offers support to teachers in the development of their instruction by assigning tutors, who provide planned and continuous support in their teaching process (Department of Education and Early
174 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Childhood Development, 2010). The functions of the tutors include the development of joint planning activities, analysis of teaching practices, record of information concerning the coaching process, and implementation of work modes that respond to the detected problems (Vezub & Alliaud, 2012). The effect of coaching in the teaching practice may vary depending on the time that the teacher is supported by the tutor, on the enforceability or voluntariness of the teacher participation, and on the content and context of coaching. SPD may have incidence on teacher´s pedagogic and teaching background by promoting the implementation of effective teaching strategies and promoting knowledge about the origin and treatment of difficulties face by students in learning specific contents (Duque, Celis, Diaz & Gomez, 2014). In Colombia, the Ministry of National Education (MEN) has executed since 2012 the program Todos a Aprender (PTA) which offers voluntary SPD to elementary school teachers in public schools nationwide, characterized by having numerous students with low academic performance. Particularly, the program assigns tutors to coach teachers of selected schools and offers work materials and specific guidelines about the procedures to conduct and goals to achieve (MEN, 2016). PTA is relevant in the Colombian context for two major reasons. First, because of the large gaps in access to quality education and in academic performance of students according to their socioeconomic characteristics (Icfes, 2016). Secondly, because of poor opportunities for restructuring the teaching staff in public schools and because is unusual that teachers are dismissed because of their low performance (OECD, 2009a). Then, although the PTA is not a comprehensive solution it is still a strategy that seeks to reduce these issues in the short term. Although there are numerous coaching programs for teachers worldwide, and different approaches to the assessment of their impact, the review of existing literature allows to conclude that the impact of PTA in Colombia has not been evaluated. Therefore, this document investigates the relationship between the participation in PTA and the students’ academic performance. Some contextual characteristics of students and schools are employed in this research, as well as the outcomes of the standardized test Saber
Revista de Educación, 381. July-September 2018, pp. 173-197 175 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
applied by the Colombian Institute for Education Evaluation (Icfes)2, and all schools’ participation in PTA for a period of five years counted from 2012, year in which the application of the program began. This data set allows to use fixed effects models in order to control per non-observable characteristics invariant over time. This permits to decrease threats to the statistical validity associated to the observational nature of the data. This document is divided as follows: after this introduction, international experiences in SPD and methods employed to evaluate their impact are summarized. Then, the operation of the PTA is described as well as the mechanism for selecting the participating schools. Further, the data and methodology employed in the empirical exercise are described. Finally, the outcomes and the discussion of the main findings are presented.
Situated Professional Development: International experiences
The importance of teachers integral development, understood as professionals´ training with the necessary competences to develop activities that foster learning, has been highlighted over the last years. Namely, what is fundamental is not the acquisition of knowledge, but the capability to transfer it and to teach how to use it through experiences (OECD, 2010). The importance of the appropriate development of this process in students’ learning has led to the implementation of SPD. This concept defines a collaborative relationship that helps teachers obtain professional learning in their place of work, which allows them to improve their actual teaching practice in the classroom (Rodriguez, Esteban, Aranda, Blanchard, Dominguez, Gonzalez, Romero, Sanz, Mamposo, Vitón & Messina, 2011). SPD programs can be classified between mandatory and voluntary, and in accordance with the type of training they offer, where we find initial training of teachers or support to low-performance teachers (Mourshed, Chijioke & Barber, 2010; Solaz-Portolés & Lopez, 2008). In some countries of Europe, Asia and Oceania there are mandatory programs addressed to beginning teachers who are instructed and coached until they acquire the capability to train other teachers (Darling-Hammond, Chung &
(2) Official institution in charge of conducting standardized tests per competences for different levels of education in Colombia.
176 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Andree, 2010). In the United States and Canada, besides the mandatory coaching programs for beginning teachers (Darling-Hammond et al., 2010; Flanagan, 2006; Lockwood, Sloan y Marsh, 2010), there are programs addressed to teachers in low performance schools (Mourshed et al., 2010; UNESCO, 2014). In Latin America, the implementation of tutoring programs for beginning teachers began in 2000 (Vezub & Alliaud, 2012). In Argentina, the National Institute for Teacher Training coordinates the coaching from higher education institutions to beginning teachers (Congreso de la Nacion Argentina, 2007). In Chile, a SPD program selects mentors trained in focalized universities to help teachers by means of joint planning, observation, and reciprocal observation of classes, feedback, and reflection upon teaching practices (Boerr, 2010). Uruguay established “pedagogic pairs” of teachers so that they provide mutual support from observation and ongoing analysis of the day-to-day practice (Rodriguez & Grilli, 2012). In Mexico and Ecuador, besides the coaching to beginning teachers, there are SPD programs for teachers who have an insufficient performance level in the teaching evaluation to improve quality on their teaching practices (Congreso General de los Estados Unidos Mexicanos, 2013; Vezub y Alliaud, 2012). Similarly, the State of Minas Gerais in Brazil developed a program with four weekly visits to schools with low performance, where principals are trained and guided so they can play the role of “coaches” for teachers (Mourshed et al., 2010).
Evaluations
Researches have evaluated the impact of SPD programs in the quality of education (Cumsille & Fitzbein, 2015). Such studies employ qualitative, quantitative, or mixed analysis methods3, focusing on the measurement of this type of programs effect in the beliefs and practices of teachers and in students´ academic performance. Qualitative research usually employs data from both the teacher and tutor’s lesson planning, sessions monitoring and surveys or interviews with the tutor, the teacher or the principal of the school (Ensminger, Kallemeyn, Rempert, Wade &
(3) Cappella, Jackson, Kim, Bilal, Hollad & Atkins (2016) and Sumner (2011) carry out mixed analysis.
Revista de Educación, 381. July-September 2018, pp. 173-197 177 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Polanin, 2015; Flanagan, 2006; Meher, Ummulbanin & Mursaleen, 2007; Neuberger, 2012). Quantitative studies, focused on programs´ impact on teaching practices, have employed multiple methodologies. Villar and Strong (2007) make a cost-benefit analysis of a mentoring program for beginning teachers; they calculate a return rate and found that after five years an investment in the program yields a positive return for the society, the school district, the teachers, the students, and the State. Bartlett (2012), Veenman, De Laat and Staring (1998), and Veenman and Denessen (2001) evaluate the teachers’ skills before and after the coaching and they find that these programs improve teaching skills by strengthening teachers’ autonomy, feedback, and encouragement to reflection. Besides, they find that mentor’s experience, duration of the tutoring, anda clear definition of the program’s objectives have a positive incidence on teachers´ performance, confidence, and aspirations (Veenman, et al., 1998; Veenman & Denessen, 2001). Researchers have estimated different models in the quantitative analysis of the effect of coaching programs on the academic performance of students. Matsumura, Garnier, Junker, Resnick and DiPrima (2009) employ a hierarchical linear model and they conclude that, although the program increases the quality of the teachers’ teaching, the students’ performance does not have a significant change. Lockwood, Sloan and Marsh (2010) estimate a fixed effects model and found a positive relationship with the students’ performance. Biancarosa, Bryk, and Dexter (2010) apply an added value model and proved that an additional year of involvement on this type of programs increases student´s learning. Yoshikawa, Snow, Barata, Gomez, Leyva, Treviño, Weiland, and Moreno (2015) use a multi-level model from which they identify positive impacts in the organization of the class and emotional and instructional support to teachers; demostrating that there is no impact on students’ socio- emotional skills or in their literacy learning.
Todos a Aprender program
PTA is a SPD addressed to public school teacher´s teaching in Elementary School level (from transition to the fifth grade) in Colombia. This program seeks to train teachers in disciplinary and didactic aspects through tutors
178 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia who coach them in the development of their lessons and work with them in additional sessions, to improve students learning conditions (MEN, 2015). Quantitatively, the program expects that more than 25% of students in focalized schools increase their performance level, at least in the areas of language and mathematics evaluated in the Saber 3, 5, and 9 (MEN, 2013). PTA involves four key components for an appropriate coaching to teachers. The pedagogic component provides curricular guidelines, educational materials in language and mathematics for teachers and students, and evaluation alternatives for learning. The basic condition component provides support in food, transportation, physical and technological infrastructure to ensure students´ permanency in the education system. The education management component enables ongoing evaluation of the program. Last, the communication, mobilization and engagement component promotes involvement of all the educational community in the improvement of learning of the students (MEN, 2011). The strategy of coaching teachers comprises trainers and tutors. Trainers are hired by the Ministry of Education and tutors are selected by the administrations of certified territorial entities (CTE). Although tutors belong to the CTE, the budget for paying the tutors and trainers comes from MEN. The PTA training processes operate in a cascade manner: trainers instruct the tutors, who in turn train the teachers in accordance with a time schedule and an agenda with defined topics and objectives (MEN, 2013). From 2017, trainers will be chosen by universities and they will coach the teachers directly. PTA is addressed to schools located in regions with notorious educational gaps, with students with difficulties in learning (MEN, 2015). Therefore, the ministry, the consulting firm McKinsey & Co., and the CTEs jointly chose the schools that would benefit from the program4. Initially MEN selected 1,616 schools regarding the most lagging schools in educational quality measured by the outcomes in Saber, with course repetition rate (above 4.6%), dropout rate (above 3.8%), and negative variations in enrolment of schools between 2009 and 2010. Additionally, it considered a regionalization component based on criteria related to socioeconomic and vulnerability indicators of the EI.
(4) The information comes from the program’s documents and several conversations and meetings with members of the MEN, PTA, and McKinsey consulting teams.
Revista de Educación, 381. July-September 2018, pp. 173-197 179 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
McKinsey & Company took into consideration enrollment and school´s accessibility in order to increase PTA´s impact in terms of coverage and cost-efficiency; consequently, it selected 1,400 more schools to be benefited. In total, this selection process resulted in 3,016 EI,out of which only 2,378 were ultimately focalized as beneficiaries of PTA. However, before the program´s implementation, some CTE requested the MEN the inclusion of some schools. In total 1,933 schools belong to this group, out of which 90% are in the category of minimum or low/priority performance. So far, the schools that began as beneficiaries of the PTA continue in the focalization. Because since the origins of the program it was expected that coaching enables the consolidation of ideas about the type of practices that teachers may achieve and sustain on their own, in 2016 the program administrators defined that the duration of the focalization will be three years (MEN, 2013). Presently the PTA benefits 2,400,000 children and 92,000 teachers in more than 4,300 institutions in 878 municipalities in Colombia (MEN, 2016).
Methodology
Representativeness of databases
Two databases are employed in this research. The first one contains the results obtained in the Saber 3, 5, and 9 tests by the third and fifth grades in each venue or school in the country. This evaluation is applied on a yearly basis by Icfes by area and grade for each schools or venue in the country. Icfes provides an average score and the percentage of students by school in accordance with four performance levels (Insufficient, Satisfactory, Minimum, and Advanced); however, on the venue level there is no average score, and results are not published on an individual student basis. Besides the evaluation, Icfes applies students’ surveys that compile information about their socioeconomic and background conditions, characteristics of their schools, and the community where school is located. Particularly, what interests most is the Socioeconomic Level (SEL) index which catalogs school into one of four socioeconomic categories, where SEL 1 is the lowest. The estimation of the SEL takes into account parents´ educational level and housing services and possessions.
180 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
The second database employed was provided by the MEN and indicates the year of entry into the program of each venue focalized between 2012 and 2016. An important characteristic for this paper is that, although the beneficiaries of PTA are schools, the focalization is effectively addressed to the venues.5 Focalization of the PTA is done on a yearly basis and it has always been carried out at the beginning of the calendar year, which matches the school year of official schools in the country. Unlike Icfes data that offers a score for each grade, area, session, and venue, the implementation of the PTA is for the entire venue. This data characteristic allows us to make separate analyses for each grade, area, and session of the venues. Each venue has a single identifier that permits to relate the databases of the MEN and Icfes, therefore, the resulting database employed in the research is by areas (mathematics or language), grades (third or fifth), and session of the venue (morning, afternoon, or full- day), and it contains information about the evaluation, the contextual characteristics, and participation in the program from 2012 to 2016. Moreover, since entry into the PTA is exclusively for public schools, the resulting database only contains venues from the official sector. Table 1 shows the amount of venues registered in the databases of Icfes and MEN, as well as the observations for which it was possible to cross- reference information from both databases. As data demonstrates, during all the years studied the percentage of venues beneficiaries of the PTA for which it was possible to link information from Saber test in one or more sessions, varies between 40% and 63% for the third grade; between 38% and 59% for the fifth grade and between 79% and 100% for both grades. Some of the reasons why it was not possible to link information from the test to some of the venues participating in the PTA include: venues without third and fifth grades, changes or problems in the venue identification code, noncompliance with the requirements that enable the publication of results, submission of answer sheets without distinction of the sheets corresponding to each venue (in those cases it is only possible to generate results at the level of school). Even though the number of observations lost in the process of matching the databases is not negligible, the amount of venues that have information about the year of entry into the program and the outcomes of the Saber test in one or more sessions is still considerably high (between 4,470 and 15,342).
(5) A venue is one of several educational facilities that belong to a single EI. There are differences among venues in terms of location, teaching staff, and population of students served. When a school with many venues is selected, only some of them are focalized due to access costs.
Revista de Educación, 381. July-September 2018, pp. 173-197 181 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
TABLE 1. Number of venues on the databases employed per year and per session.
Venues with outcomes in Cruce Saber tests PTA 3rd Grade 5th Grade 3rd Grade 5th Grade Morning 2012 15,402 14,130 9,517 3,681 3,399 2013 12,170 11,376 23,166 6,129 5,749 2014 14,481 13,541 24,045 7,774 7,202 2015 15,185 14,227 25,466 8,552 7,958 2016 19,080 17,867 24,408 10,622 9,864 Afternoon 2012 2,348 2,253 9,517 743 714 2013 2,027 1,944 23,166 1,026 1,012 2014 2,625 2,504 24,045 1,415 1,380 2015 2,592 2,482 25,466 1,479 1,422 2016 2,815 2,678 24,408 1,569 1,506 Full-day 2012 6,979 6,622 9,517 379 357 2013 6,104 5,897 23,166 2,194 2,149 2014 6,777 6,638 24,045 2,552 2,510 2015 6,237 6,071 25,466 2,498 2,476 2016 8,176 7,934 24,408 3,151 3,086
* Note: Implementation of the PTA is done per venues and not per sessions, therefore, it is not possible to establish which sessions are focalized by the program. A single venue may have different sessions. Source: author
Some characteristics evinced by the data
Table 2 shows the frequency of official venues which pertain and do not pertain to the PTA per SEL. Venues reported as N/A are not catalogued in a SEL due to difficulties when completing the survey. The table shows the PTA focalization growth in the years of the study: whereas in 2012 the program served almost 20% of the official sector venues, in 2016 it already coached more than half of those. Both the PTA focalization and its growth over time are concentrated in the venues of lowest SEL, especially in SEL 2. In 2016, the venues of SEL 1 represent little under 45% of the venues focalized by the program while the ones of SEL 1 and 2, 98%.
182 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
TABLE 2. Number of official venues with and without PTA, per SEL and year.
Year No Sí 2012 9,732 2,205 2013 4,716 3,919 SEL 1 2014 5,320 4,828 2015 5,060 5,577 2016 7,250 7,039 2012 10,095 2,511 2013 5,558 5,375 SEL 2 2014 6,129 6,782 2015 5,716 6,799 2016 7,075 8,393 2012 966 109 2013 802 209 SEL 3 2014 951 280 2015 916 275 2016 988 299 2012 16 0 2013 20 0 SEL 4 2014 16 0 2015 20 0 2016 19 0 2012 228 13 2013 163 11 N/A 2014 79 11 2015 80 6 2016 81 7 2012 21,037 4,838 2013 11,259 9,514 Total 2014 12,495 11,901 2015 11,792 12,657 2016 15,413 15,738
Source: author
Revista de Educación, 381. July-September 2018, pp. 173-197 183 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Table 2 shows that even though the program is focused on schools of low SEL, the focalization is not strictly restricted to this type of schools. This circumstance is given by different reasons. Acceptance and recognition of the program has made it grow over time, and that more schools want to participate. The regional administrations are intermediary in the process and may have incentives to postulate certain schools. The PTA is not a mandatory program, so principals of pre-selected institutions may decide not to participate. Since the selection of program beneficiaries may not be considered random, it is not simple to estimate the impact of PTA on the students’ academic performance. However, the aforementioned circumstances about the selection of participating venues provide a favorable scenario for the analysis (or identification strategy) because they allow comparing relatively similar venues with and without PTA. Table 3 shows the percentage of students in official venues who, in average, are in the Insufficient and Minimum performance levels, with and without PTA for the third grade in mathematics in the morning session. Three main characteristics are notable in these data: there are more venues in the low SEL than in the upper ones; as the SEL decreases, the percentage of students increases in the two lowest performance levels, that is to say, the students’ academic performance decreases and both the venues beneficiaries of the PTA and the ones which do not, seem to improve over time.
184 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
TABLE 3. Percentage of third grade students in official venues, mathematics, morning session, in the Insufficient and Minimum performance levels with and without PTA, per SEL and year6
Without PTA With PTA N* Mean Std. Dev. N Mean Std.Dev. 2012 5,187 49.88 43.27 1,626 63.75 40.39 2013 2,271 43.17 42.57 2,338 65.18 39.81 SEL 1 2014 2,779 49.12 42.77 2,984 62.14 40.10 2015 2,716 35.81 41.86 3,611 54.28 42.19 2016 3,739 36.74 42.87 4,474 43.57 42.02 2012 4,791 49.67 36.60 1,737 62.64 33.47 2013 2,473 47.91 36.25 3,099 56.28 35.76 SEL 2 2014 2,677 46.33 35.15 4,008 53.61 34.88 2015 2,639 40.77 36.52 4,032 51.79 36.19 2016 3,116 42.91 37.42 4,906 44.45 36.96 2012 628 39.39 20.52 67 43.06 21.77 2013 521 34.90 19.15 131 38.34 22.48 SEL 3 2014 596 32.79 19.28 191 37.81 20.68 2015 580 35.28 19.87 186 39.20 22.39 2016 577 42.40 10.84 191 43.57 23.31 2012 7 16.15 15.49 2013 10 28.53 22.74 SEL 4 2014 7 8.27 4.23 2015 9 13.79 10.04 2016 8 28.65 14.94 2012 125 46.99 44.92 9 69.53 36.37 2013 81 42.41 47.07 4 56.81 41.72 SEL N/A 2014 51 36.60 45.59 9 46.30 44.70 2015 58 21.33 39.44 5 40.00 54.77 2016 56 19.35 38.67 6 16.67 40.82 Total 35,702 33,614
* Number of venues where the reported percentage of students is located. Source: author
Methods
Fixed effects models are used at venue level for each session, grade and area in order to attempt estimating the impact of the PTA in the
(6) The trend of the results is similar for the other combinations among grades, areas, and sessions.
Revista de Educación, 381. July-September 2018, pp. 173-197 185 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
students’ performance in a context of non-randomness in the selection of the treatment, as follows:
yjt=f (β0+β1+PTAjt+β2Xjt+δi+µit) (1)
Where f (.) is a linear function or Logit in which j = (1, 2… N) is the session, area, and grade of a venue, and t = (2012, 2013, 2014, 2015,
2016) is the year for which results of Saber test are available. yjt is the
outcome of the students’ evaluation in the venue j in the year t. PTAjt = (0, 1, 2, 3, 4, 5) is a variable that counts the number of participating years of a venue in the PTA7; in this way, a venue that has never been in a program has a value of zero and one that has been since 2012 has a value
of one in 2012, two in 2013, up to the value of five in 2016. Moreover,X jt 8 compiles the characteristics of the venue which vary over time ; δi are the observable and non-observable invariant characteristics over time, and is
the term of error. Last, β1 is the coefficient of interest that, given a good specification of the model, shows the effect that an additional year has on the average performance of the students in the venue. The choice of a fixed effects model is due to the existing correlation between the non-observable characteristics invariant over time and the participation in PTA, explained by the non-randomness in the selection of beneficiaries of the program. For example: the students’ SEL and the location of the venue are observable characteristics correlated to the PTA, whereas the willingness of the principal to participate in the program and the teachers´ and students´ intrinsic skills may be non-observable characteristics. If fixed effects were not taken into account at the venue level (correlated to the participation in the program), these would be
included in the term of error and β1 would be biased. Since the implementation of PTA takes place in the venues9, the percentage of students per levels of performance is employed in order to construct two response variables. The first one is the sum of the percentage of students in the Insufficient and Minimum performance levels, which may be considered an appropriate variable by considering that PTA´s
(7) The specification of the PTA variable allows to analyze the intensity of the effect and have available multiple controls and treatments with similar characteristics. (8) Most of the demographic and socioeconomic characteristics compiled by Icfes vary Little over time; therefore, Xjt is only the number of students evaluated in the session, grade, and area of the venue. (9) Icfes only reports the average score at school level in the Saber 3, 5, and 9 tests.
186 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia objective is to reduce the quantity of children in the lowest performance levels. The second dependent variable is a dummy that indicates whether the venue has increased the percentage of children in the lowest levels of the distribution from one year to another; therefore, it takes the value of one if the difference from one year to another between the sum of the percentage of students in the Insufficient and Minimum performance levels is greater than zero; otherwise it takes the value of zero. A linear model which attempts to estimate the effect that an additional year of participation in PTA has on decreasing the percentage of students in the Insufficient and Minimum performance levels is estimated with the variable in levels. A Logit model that shows the direction of the effect that an additional year of the PTA has on the probability of increasing the percentage of students in the lowest levels of the distribution is estimated with the dummy variable. There are three observations in the Logit model per venue because of the difference-dependent variable. The models observations only include the venues that belong to the SEL 1 and 2 in order to analyze venues that have a similar likelihood of having been selected as beneficiaries of the program. Two specifications are made for each combination of areas, grades, and sessions: the first one includes all public venues of the group, and the second one considers only those that have participated in PTA at least for one year. This exclusion is done because the venues that have not participated in the program in any of the five years might have non-observable characteristics systematically different to the ones that have participated. Table 4 shows the amount of official venues in the SEL 1 and 2 for the two specifications mentioned above. It may be understood as a filter of Table 1 since both specifications are obtained by selecting the official venues in SEL 1 and 2 from Icfes database that contains all venues for which it was possible to cross-reference information of databases from Icfes and MEN.
Revista de Educación, 381. July-September 2018, pp. 173-197 187 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
TABLE 4. Number of observations in each specification, per year and session.
Venues with published results of Cross-reference Saber tests SEL 1&2 SEL 1&2 3rd Grade 5th Grade 3rd Grade 5th Grade Morning 2012 14,533 13,346 3,602 3,320 2013 11,389 10,650 5,993 5,616 2014 13,602 12,763 7,567 7,016 2015 14,325 13,495 8,356 7,783 2016 18,212 17,099 10,421 9,676 Afternoon 2012 1,792 1,714 686 665 2013 1,509 1,455 934 927 2014 1,977 1,904 1,283 1,259 2015 1,962 1,871 1,343 1,293 2016 2,146 2,053 1,424 1,373 Full-day 2012 6,850 6,498 376 355 2013 5,970 5,765 2,172 2,130 2014 6,652 6,516 2,536 2,497 2015 6,098 5,939 2,478 2,457 2016 7,987 7,746 3,123 3,059
Source: author
Results
As mentioned, there is a non-negligible number of venues beneficiary of the PTA for which it was not possible to link information of the Saber 3, 5, and 9 tests. Consequently, the results shown below are valid for the venues for which we have information of the year of entry to the program and the outcomes of the test.
188 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Effect of the PTA on the percentage of students in the Minimum and Insufficient performance levels
In the linear model, the coefficient of interest represents the average marginal effect of one additional year in the program over the percentage of students in the Insufficient and Minimum performance levels. This percentage, conditioned by the invariant characteristics over time of venues and the number of students evaluated each year allows to compare the effect among the grades, areas and sessions considered. The results (Table 5) indicate that an additional year in the PTA is correlated with an average decrease of the percentage of students in the lowest performance levels, that is to say, the PTA is associated to an average improvement in the students’ performance in the grades and areas considered, where third grade in mathematics shows the greatest impact. Although the functional form of the linear model may be inappropriate since the dependent variable is a percentage, it is employed because the results of Saber have a normal distribution and the amount of venues without students on both lowest performance levels or with all students in those levels is minimum.
TABLE 5. Coefficient of the PTA in the linear model per grade, area, and session.
LANGUAGE MATHEMATICS Third Fifth Third Fifth Only PTA Official Only PTA Official Only PTA Official Only PTA Official Full-day session Years in PTA -4.44*** -3.51*** -2.01*** -1.30*** -4.28*** -3.33*** -2.35*** -2.71*** (-9.75) (-10.09) (-5.10) (-4.33) (-9.41) (-9.85) (-5.77) (-8.84) N 9,510 29,807 9,806 30,187 9,541 29,771 9,753 29,918 Morning session Years in PTA -4.50*** -4.23*** -2.44*** -2.05*** -5.23*** -4.67*** -3.02*** -3.05*** (-22.65) (-24.48) (-14.22) (-13.61) (-25.05) (-26.13) (-17.54) (-20.27) N 32,622 65,052 31,837 63,635 32,815 65,203 31,675 63,233 Afternoon session Years in PTA -2.65*** -2.84*** -2.13*** -2.02*** -3.26*** -3.14*** -2.29*** -2.36*** (-9.04) (-10.88) (-10.02) (-10.48) (-10.25) (-10.95) (-10.30) (-11.66) N 5,480 9,129 5,495 8,965 5,548 9,208 5,488 8,957 t-statistic shown in parentheses * p<0,05 ** p<0,01 *** p<0,001 Source: author
Revista de Educación, 381. July-September 2018, pp. 173-197 189 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Effect of the PTA on the likelihood of increasing the percentage of students in the Minimum and Insufficient levels
A Logit model was estimated to know the direction that the effect of an additional year of PTA has on the likelihood to increase the percentage of students in the lowest levels. In this case the response variable has the form of added value so that it allows comparing the change from one year to another among the PTA participant and non-participant venues. Measurements in the form of added value have well-known advantages over the measures in levels because they allow to improve the observational condition of the standardized tests by comparing the changes, not only the output conditions (Downes & Vindurampulle, 2007; Drury & Doran, 2003; OECD, 2008; Willms, 2009). As, in average, the venues focalized in the PTA have a lower performance than the non- focalized, this may be a good specification. Table 6 shows the results of the Logit model and evinces that the effect is negative and significant for all grades, subjects, and sessions.
TABLE 6. PTA coefficient in the Logit model per grade, area, and session.
LANGUAGE MATHEMATICS Third Fifth Third Fifth Only PTA Official Only PTA Official Only PTA Official Only PTA Official Full-day session Years in PTA -0.23*** -0.18*** -0.10* -0.06* -0.25*** -0.17*** -0.13** -0.18*** (-6.86) (-6.99) (-2.46) (-2.00) (-7.72) (-7.00) (-3.24) (-5.84) N 3,961 14,705 3,081 12,049 4,274 15,510 3,060 11,096 Full-day session Years in PTA -0.27*** -0.24*** -0.18*** -0.14*** -0.34*** -0.29*** -0.26*** -0.26*** (-12.25) (-13.00) (-6.01) (-5.77) (-16.58) (-16.63) (-8.78) (-10.64) N 9,146 23,225 5,775 16,312 10,507 25,636 5,505 15,172 Full-day session Years in PTA -0.18 -0.24* -0.44 -0.40 -0.30** -0.23* -0.62* -0.67* (-1.44) (-2.19) (-1.54) (-1.66) (-2.68) (-2.34) (-2.06) (-2.40) N 271 514 60 102 329 622 55 82
t-statistical shown in parentheses * p<0,05 ** p<0,01 *** p<0,001 Source: author
190 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
In general, the models employed allow to conclude that for the third and fifth grades in mathematics, and for the third grade in language, the PTA is associated to an improvement in the outcomes of the students. Besides, for all grades and areas of the full-day and morning sessions, the PTA is associated to a decrease in the likelihood of not improving.
Robustness
The estimation of the linear and the Logit models was carried out along with some verifications of robustness. Annexes 1 and 2 show the linear model estimated in an independent manner for SEL 1 and 2. This exercise, which allows to compare controls and treatments with similar characteristics, was also done with the Logit model where the result expected was found in all grades, areas, and sessions. The annexes evince that the results on Table 5 are maintained for the case of SEL 1 and that the magnitude of the effect is larger. This finding may be interesting for decision makers on public policies since it suggests that resources are more efficient if they are addressed to the poorest population. However, it is important to remember that the average performance of SEL 1 venues, both without and with PTA, is considerably lower than in other SEL, therefore the need for improvement is much bigger.
Conclusions
Most significant findings and public policy recommendations
This research estimates the impact of the PTA on the average skills of students in third and fifth grade measured through the Saber 3, 5, and 9 tests. The results show that for most of grades and areas studied there is a partial, positive correlation between the students’ performance and an additional year of participation in the program. However, it is important to clarify that the channels through which the program operates were not analyzed. Hence, it is found that in average, the institutions focalized by the PTA are moderately improving more than non-focalized ones, but the reasons to explain this difference are unknown.
Revista de Educación, 381. July-September 2018, pp. 173-197 191 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Additionally, the results evince that the impact is stronger on schools that have been focalized for more time and on the ones in the lowest SEL, which suggests that the program has increasing returns and is more efficient in schools of lower SEL. These findings are useful for public policies and decision makers with regards to continuing and expanding the program, improving the coaching strategies, streamlining the mechanism for selection of the institutions, and prioritizing the focalization of institutions of the lowest SEL.
Limitations and possibilities for improvement
It is important to recognize that these estimations may be biased due to non-observable characteristics that vary over time and because of common characteristics between the observations for which there is information about the participation in the program and the test’s outcomes. Even so, considering the high number of venues beneficiary of the program for which it was possible to link the test’s results, and under the assumption that there have not been considerable changes over time related to PTA which affect systematically the performance of the students, these results suggest that the program is generating a modest yet perceptible improvement in the students’ performance. Also, because the observational nature of the data these results must be read with caution. The empirical approach employed may be improved and complemented in different manners. The Ministry of Education is consolidating a database of administrative records that comprise the number of visit to classrooms, sessions of tutors with the trainers and teachers, and materials handed out. Besides, it is classifying the tutors qualitatively on the basis of some performance criteria. The interaction of these variables with the participation in the program may enable the understanding of the mechanism by which the PTA is positively correlated with the students’ performance and allow the definition of specific recommendations about ways to intervene. Last, we considered that there are some important questions unanswered about the possible implications of PTA in the students` learning process. Further research may evaluate whether the benefits of the program exceed their costs and explore the mechanisms by which the
192 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia participation in the program may lead to improvements in the students’ academic performance.
Bibliographic references
Bartlett, J. (2012). A model role - Evaluation of the Mosaic mentoring programmes. London: Demos. Barrera, F., Maldonado, D. y Rodríguez, C. (2012). Calidad de la educación básica y media en Colombia: Diagnóstico y propuestas. Serie documentos de trabajo, 126. Biancarosa, G., Bryk, A. y Dexter, E. (2010). Assessing the value-added effects of literacy collaborative professional development on student learning. The elementary school journal, 111(1), 7 – 34. Boerr, I. (2010). Acompañar los primeros pasos de los docentes. Chile: Santillana. Cappella, E., Jackson, D., Kim, H., Bilal, C., Hollad, S. y Atkins, M. (2016). Implementation of teacher consultation and coaching in urban schools: A mixed method study. School Mental Health, 8(2), 222 – 237. Cid, A., Pérez, A. y Sarmiento, J. (2011). La tutoría en el Practicum. Revisión de la literatura. Revista de Educación, 354, 127 – 154. Congreso de la Nación Argentina. (2007). Ley de Educación Nacional 26.206. Disponible en: http://www.me.gov.ar/doc_pdf/ley_de_educ_ nac.pdf Congreso General de los Estados Unidos Mexicanos. (2013). Ley General del Servicio Profesional Docente. Disponible en: http://www.dof.gob. mx/nota_detalle.php?codigo=5313843&fecha=11/09/2013 Cumsille, B. y Fiszbein, A. (2015). Construyendo políticas docentes efectivas: Pautas para la acción. Comisión para la Educación de Calidad para todos: Documento de antecedentes. Darling-Hammond, L., Chung, R. y Andree, A. (2010). How High-Achieving Countries Develop Great Teachers. Stanford, CA: Stanford Center for Opportunity Policy in Education. Department of Education and Early Childhood Development. (2010). Coaching Teachers in Effective Instruction. Melbourne: Student Learning Division Office for Government School Education.
Revista de Educación, 381. July-September 2018, pp. 173-197 193 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Downes, D. y Vindurampulle, O. (2007). Value-added Measures for School Improvement. Melbourne: Department of Education and Early Childhood Development. Duque, M., Celis, J., Diaz, B. y Gomez, M. (2014). Diez pilares para un programa de desarrollo profesional docente centrado en el aprendizaje de los estudiantes. Revista Colombiana de Educación, 67, 107 – 124. Drury, D. y Doran, H. (2003). The Value of Value-Added Analysis. Policy Research Brief. National School Boards Association, 3(1), 1 – 4. Ensminger, D., Kallemeyn, L., Rempert, T., Wade, J. y Polanin, M. (2015). Case study of an evaluation coaching model: Exploring the role of the evaluator. Evaluation and Program Planning, 49, 124 – 136. Flanagan, T. (2006). The perceived effectiveness of a beginning teacher mentoring program in central Virginia. (Tesis inédita de doctorado). Liberty University. García, S. (1999). El desarrollo profesional: análisis de un concepto complejo. Revista de Educación, 318, 175 – 187. Icfes. (2016). Informe Nacional Saber 3°, 5° y 9°. Resultados Nacionales 2009 – 2014. Lockwood, J., Sloan, J. y Marsh, J. (2010). Linking reading coaches and student achievement: Evidence from Florida middle schools. Educational Evaluation and Policy Analysis, 32(3), 372 – 388. Matsumura, L., Garnier, H., Junker, B., Resnick, L. y DiPrima, D. (2009). The influence of content-focused coaching on reading comprehension instruction and Student Achievement. SREE Conference abstract template. Meher, R., Ummulbanin, A. y Mursaleen, G. (2007). Teacher mentoring programme: A vehicle to support professional development for improving the quality of education in Districts of Sindh and Balochistan of Pakistan. Quality in Education: Teaching and Leadership in Challenging Times, 354 – 366. MEN. (1979). Decreto N°2.277 de 1979. Estatuto docente. Septiembre 14 de 1979. MEN. (2011). Programa para la transformación de la calidad educativa: Sustentos del programa. MEN. (2013). Orientaciones técnicas para la producción de secuencias didácticas para un Desarrollo Profesional Situado en las áreas de MATHEMATICS y ciencias. Bogotá.
194 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
MEN. (2015). ¿Qué es Todos a Aprender? Obtenido de Colombia Aprende: http://aprende.colombiaaprende.edu.co/es/pta/87535 MEN. (2016). Cifras: Colombia Aprende. Obtenido de Colombia Aprende: http://aprende.colombiaaprende.edu.co/es/pta/87542 Mourshed, M., Chijioke, C. y Barber, M. (2010). How the world´s most improved school systems keep getting better. London: McKinsey & Company. Neuberger, J. (2012). Benefits of a teacher and coach collaboration: A case study. The Journal of Mathematical Behavior, 31(2), 290 – 311. OCDE. (2008). Measuring improvements in learning outcomes: Best practices to assess the value-added of schools. Paris: OECD. OCDE. (2009 a). Los docentes son importantes: atraer, formar y conservar a los docentes eficientes.OECD Publishing. OCDE. (2009 b). Evaluación y reconocimiento de la calidad de los docentes. Prácticas internacionales. OECD Publishing. OECD. (2010). OECD Reviews of Human Resource Management in Government: Brazil 2010: Federal Government, OECD Publishing, Paris. http://dx.doi.org/10.1787/9789264082229-en Rodríguez, A., Esteban, R., Aranda, R., Blanchard, M., Domínguez, C., González, P., Romero, P., Sanz, E., Mampaso, A., Vitón, M. y Messina, C. (2011). Coaching reflexivo entre iguales en el Practicum de la formación de maestros. Revista de Educación, 355, 355 – 379. Rodríguez, E. y Grilli, J. (2012). Transitar hacia la profesión docente. La pareja pedagógica como dispositivo en la formación inicial de profesores para la educación media en Uruguay. III Congreso Internacional sobre profesorado principiante e inserción profesional a la docencia, Santiago de Chile. Sánchez, M. y Corte, F. (2015). La evaluación a la docencia: Algunas consecuencias para América Latina. Revista Mexicana de Investigación Educativa, 20(67), 1233 – 1253. Solaz-Portolés, J. y López, V. (2008). Types of knowledge and their relations to problem solving in science: directions for practice. Educational Sciences Journal, 6, 105-112. Sumner, K. (2011). An explanatory mixed-methods study of instructional coaching practices and their relationship to student achievement. (Tesis inédita de doctorado). Western Carolina University. UNESCO. (2014). Temas críticos para formular nuevas políticas docentes en América Latina y el Caribe: el debate actual. Santiago: OREALC/ UNESCO Santiago.
Revista de Educación, 381. July-September 2018, pp. 173-197 195 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Veenman, S., De Laat, H. y Staring, C. (1998). Evaluation of a coaching programme for mentors of beginning teachers. Journal of In-Service Education, 24(3), 411 – 431. Veenman, S. y Denessen, E. (2001). The coaching of teachers: results of five training.Educational Research and Evaluation, 7(4), 385 – 417. Vezub, L. y Alliaud, A. (2012). El acompañamiento pedagógico como estrategia de apoyo y desarrollo profesional de los docentes nóveles: Aportes conceptuales y operativos para un programa de apoyo a los docentes principiantes de Uruguay. OEI. Villar, A. y Strong, M. (2007). Is mentoring worth the money? A benefit- cost analysis and five-year rate of return of a comprehensive mentoring program for beginning teachers. Educational Research Service, 25(3), 1 – 17. Willms, D. (2009). Value Added Models in Education. Policy Brief. Canadian Research Institute for Social Policy. Yoshikawa, H., Snow, C., Barata, M., Gomez, C., Leyva, D., Treviño, E., Weiland, C. y Moreno, L. (2015). Experimental impacts of a teacher professional development program in Chile on Preschool Classroom quality and child outcomes. Developmental Psychology, 51(3), 309 – 322.
Contact Address: Ximena Dueñas Herrera. Instituto Colombiano para la Evaluación de la Educación, ICFES. Bogotá. Carrera 4, nº 26ª-42, Apartamento 317, Bogotá. E-mail: [email protected]
196 Revista de Educación, 381. July-September 2018, pp. 173-197 Received: 30-08-2017 Accepted: 23-03-2018 Dueñas Herrera, X., Escobar Escobar, A.B., Godoy Mateus, S., Duarte Rodríguez, J. L. Linking coaches and student achievement: PTA in Colombia
Annexes
ANNEX 1. Linear model only for SEL 1.
LANGUAGE MATHEMATICS Third Fifth Third Fifth PTA only Official PTA only Official PTA only Official PTA only Official Full-day session Years in PTA -5.92*** -4.79*** -2.12*** -1.97*** -6.51*** -4.93*** -2.19*** -2.99*** (-7.43) (-8.25) (-3.24) (-4.08) (-8.55) (-8.55) (-3.21) (-5.79) N 3,879 12,197 4,023 12,394 3,881 12,146 3,993 12,240 Morning session Years in PTA -5.46*** -4.92*** -2.99*** -2.61*** -7.18*** -6.19*** -3.69*** -3.69*** (-16.05) (-16.86) (-10.42) (-10.55) (-19.88) (-20.46) (-13.27) (-15.51) N 14,932 31,594 14,554 30,951 15,033 31,725 14,470 30,726 Afternoon session Years in PTA -5.40*** -5.00*** -2.85*** -3.07*** -7.43*** -7.07*** -3.22*** -3.28*** (-4.85) (-5.24) (-4.30) (-5.17) (-6.85) (-7.22) (-4.45) (-5.00) N 599 765 628 784 618 787 625 780 t-statistical shown in parentheses * p<0,05 ** p<0,01 *** p<0,001 Source: author
ANNEX 2. Linear model only for SEL 2.
LENGUAJE MATEMÁTICAS Tercero Quinto Tercero Quinto Solo PTA Oficiales Solo PTA Oficiales Solo PTA Oficiales Solo PTA Oficiales Jornada Completa Años PTA -3.59*** -2.72*** -1.94*** -0.91* -2.97*** -2.35*** -2.43*** -2.56*** (-6.50) (-6.29) (-3.94) (-2.37) (-5.26) (-5.45) (-4.81) (-6.73) N 5,631 17,610 5,783 17,793 5,660 17,625 5,760 17,678 Jornada Mañana Años PTA -3.81*** -3.72*** -2.04*** -1.64*** -3.85*** -3.56*** -2.54*** -2.58*** (-15.96) (-17.72) (-9.70) (-8.75) (-15.53) (-16.51) (-11.64) (-13.35) N 17,690 33,458 17,283 32,684 17,782 33,478 17,205 32,507 Jornada Tarde Años PTA -2.37*** -2.61*** -2.05*** -1.92*** -2.81*** -2.72*** -2.19*** -2.27*** (-7.84) (-9.68) (-9.14) (-9.41) (-8.50) (-9.12) (-9.36) (-10.67) N 4,881 8,364 4,867 8,181 4,930 8,421 4,863 8,177 t-statistical shown in parentheses * p<0,05 ** p<0,01 *** p<0,001 Source: author
Revista de Educación, 381. July-September 2018, pp. 173-197 197 Received: 30-08-2017 Accepted: 23-03-2018
State of scientific practices and educational research. Potential challenges for the next decade
Estado de las prácticas científicas e investigación educativa. Posibles retos para la próxima década
DOI: 10.4438/1988-592X-RE-2017-381-386
Ángeles Blanco-Blanco Universidad Complutense de Madrid
Abstract This paper presents a review of the state of current scientific practices and their potential impact on the quality of educational research. From a post positivist conception of scientific research in education, the matter is addressed inthe general context of the current debate about science, its reliability, robustness and reproducibility. Theoretically and conceptually, the study adopts a meta- research approach. From a methodological perspective, a review of the literature on the subject is carried out that allows a reasoned reflection on the status quo of scientific practices. This is done by considering some of the most relevant papers published in recent years on meta-science in general, and in the field of Education and Behavioral Sciences. In the first place, the so-called crisis of science is characterized, and particularly the presence of biases and questionable research practices in scientific research. Below are some of the key corrective elements proposed to strengthen and enable a more effective advancement of the scientific enterprise. These include: alternative and “new” emphasis on the statistical analysis of scientific data; renewed impetus to replication and reproducibility; and new modes of production, dissemination and assessment of research associated with open science. The article closes with some reflections regarding possible challenges for Spanish educational research in the next decade. The conclusions are organized around four axes: the development of meta-research studies; training, information and awareness-raising of researchers about questionable research practices; updating editorial policies; and the role of funders and evaluators of scientific production.
Revista de Educación, 381. July-September 2018, pp. 199-221 199 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
Key words: Questionable research practices, Reproducibility crisis, Open Science, Scientific ecosystem, Meta-research, Educational research.
Resumen El trabajo presenta una revisión del estado de las prácticas científicas actuales y de su impacto potencial en la calidad de la investigación educativa. Desde una concepción post-positivista de la investigación científica en educación, el problema se desarrolla en el contexto general del debate actual sobre la ciencia, sobre su fiabilidad, su robustez y su reproducibilidad. Teórica y conceptualmente el estudio adopta el enfoque propio de la meta-investigación. Desde el punto de vista metodológico, se lleva a cabo una revisión de la literatura sobre el tema que permita una reflexión fundamentada sobre el status quo de las prácticas científicas. Para ello se consideran algunos de los trabajos más relevantes y de mayor impacto publicados en los últimos años sobre meta-ciencia, en general y en el ámbito de las Ciencias de la Educación y del Comportamiento en particular. En primer lugar se caracteriza lo que se ha etiquetado como crisis de la ciencia y particularmente se trata de la presencia de sesgos y prácticas de investigación cuestionables en la investigación científica. A continuación se presentan algunos de los elementos correctores clave que se han propuesto para fortalecer y permitir un avance más efectivo de la empresa científica. Estos incluyen: alternativas y “nuevos” énfasis en el análisis estadístico de datos científicos; el impulso renovado a la replicación y la reproducibilidad; y los nuevos modos de producción, difusión y evaluación de la investigación ligados a la ciencia abierta. El artículo se cierra con algunas reflexiones relativas a posibles retos para la investigación educativa española en la próxima década. Las conclusiones se organizan en torno a cuatro ejes: el desarrollo de estudios meta-investigativos; la formación, información y sensibilización de los investigadores sobre prácticas de investigación cuestionables; la actualización de las políticas editoriales; y el papel de los financiadores y evaluadores de la producción científica.
Palabras clave: Prácticas de investigación cuestionables, Crisis de reproducibilidad, Ciencia abierta, Ecosistemas de la producción y la publicación científicas, Meta-investigación, Investigación educativa.
Empirical research in education is required to inform about educational practices and interventions. However, in order to be able to effectively contribute to the decision-making, one necessary condition, although not the only one, is that the educational research be scientifically robust, reliable and valid. This study aims to provide a systematic reflection on this condition. We present a review of current scientific practices
200 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade to study their potential impact on the quality and rigor of scientific research. The analysis is carried out from a post-positivist conception of scientific research in education, and in the general context of the current debate about science, its reliability, robustness and reproducibility. From a theoretical-conceptual perspective the study adopts a meta-research approach. From a methodological perspective, a literature review was conducted on the study area which permitted a well-founded reflection on the status quo of current scientific practices. This was carried out by considering some of the most relevant and highest impact studies published in recent years on meta-science in general, and more specifically on Educational and Behavioral Sciences.
Science in crisis and meta-research
In recent years, a wide range of disciplinary fields have been experiencing a crisis of confidence in what could be referred to as the status quo of scientific practices. There has been talk of acrisis of science and, especially, a crisis of reproducibility (Munafò et al., 2017). This situation is reflected by articles simultaneously published in the journals Science and Nature in 2014 under the revealing title: Journals unite for reproducibility (Nature, 2014; McNutt, 2014). The publishers describe a joint agreement to adopt new principles and guidelines for the publication of clinical research. However, fundamentally, and the point we would like to emphasize here, is that their ultimate goal was to encourage recent initiatives to respond to this general lack of trust arising from evidence of poor reproducibility, fraud and malpractice, which has been undermining the trust of experts, and the general public alike, in science. The scientific community’s concern about this is illustrated by the results of a recent survey published in Nature (Baker, 2016). Of the 1576 researchers completing the questionnaire, 90% considered a reproducibility crisis to exist and 52% regarded it to be significant. The causes most frequently mentioned included questionable research practices (e.g. selective reporting or p-hacking), the pressure to publish and matters related to use of the statistical system applied and the design of the research. This last statistical-methodological dimension heads the list of solutions mentioned by the survey respondents, especially in relation to improving training in, and supervision of, statistics and
Revista de Educación, 381. July-September 2018, pp. 199-221 201 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
methodology. In fact, it was mentioned that the crisis could be largely due to an incorrect use of statistics in scientific research (Goodman, 2016; Peng, 2015). The difficulties encountered to reproduce the published work should first be regarded as a symptom, as the result of a complex problem linked to several factors associated with the setting and the parameters with which the science is generated and disseminated. As Makel and Plucker remind us (2014), the fact that a result is not reproducible does not make it false. Moreover, a reproducible result is not necessarily a correct one. However, reproducibility is a cornerstone of the scientific system and the debate surrounding it seems to have become focused, and rightly so, on the need for rigor and transparency in scientific practices1. This crisis has possibly been the most turbulent in recent years in the area of Psychology (Ledgerwood, 2014), an area that shares similarities with that of Education. The detection of fraudulent work conducted by respected researchers, the publication of dubious and controversial discoveries and evidence of questionable practices have all fueled the suspicion that perhaps all is not well behind the high success rates of researchers’ predictions for results from their own studies. Hence, Simmons, Nelson & Simonsohn (2011) suggested that the hyper-precision in the prediction (e.g. rejection of the null hypothesis) is probably due to several factors including: collecting data only until the desired results have been reached, not reporting information about failed experiments, omission a posteriori of observations or variables that do not support the proposed hypothesis. In other words, a long list of questionable research practices. All this has given rise to an extensive and lively debate in the area of psychology research in recent years. Although we do not give a detailed description here, this is well-illustrated by at least two indicators:
■ The publication of numerous special sections on questionable research practices, replicability and reproducibility in renowned
(1) Indeed replication, understood as the repetition of previous studies by independent researchers, is the essential approach to collect scientific evidence that supports or refutes a study hypothesis. Replicability, therefore, is associated with the probability that an independent study produces results consistent with the original study. The concept of reproducibility is a recent variation in this concept (Peng, 2015). It refers to a minimum standard relating to research that entails the massive computation of data. It is defined as the possibility of recalculating the original results given the database and the original analytical specifications (e.g. code and documentation).
202 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
journals such as Perspectives on Psychological Science (2012, 2014) or Psykometrica (Sijtsma, 2016; Sijtsma, Veldkamp & Wicherts, 2016; Waldman & Lilienfeld, 2015; Wigboldus & Dotch, 2015). ■ The encouragement of ambitious large-scale research programs specifically aimed at replicating the results of published psychology research, mainly adopting new collaborative work approaches between independent research teams and consortia (see Open Science Collaboration, 2015 or Schweinsberg et al., 2016).
In our opinion, this crisis is less evident in educational research, which does not mean that similar problems are not encountered. This could be due to a lack of scientific evidence of these occurrences or the existence of fewer data to support them, but the concern is still there. This is illustrated by two main indicators: First, the publication of a recent pioneering and highly ambitious work on replication in Education Sciences (Makel & Plucker, 2014); Secondly, the launch of a new journal by AERA called AERA Open. In the editorial of its first issue, the journal defines itself as being open to inspection and replication, echoing some of the elements associated with the debate on transparency and reproducibility that we will return to later (Warschauer, Duncan & Eccles, 2015). A useful framework within which to contemplate all these problems is that of meta-research, an emerging discipline or study area that approaches science and its practices from a bird’s eye view, in other words, by adopting a broader and more comprehensive perspective (Ioannidis, Fanelli, Dunne & Goodman, 2015). Meta-research focuses on how scientific research is conducted, communicated, verified, evaluated and promoted, and focuses on different aspects of each of these areas (see table 1). Here, we will examine some of these, especially those related to questionable research practices (in the area of methods), reproducibility and new practices in the publication and dissemination of scientific research (in the area of reports), mainly focusing on Social, Behavior and Education sciences. We will also make some observations about evaluation and recognition processes in the conclusions section.
Revista de Educación, 381. July-September 2018, pp. 199-221 203 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
TABLE I. Meta-research: subject areas
Area Specific examples of interest Methods Biases and questionable practices in implementation of the research, methods (implementation) to reduce such biases, meta-analysis, synthesis of the research, integration of evidence, scientific consortia and collaborative research teams, research integrity and ethics. Reports Biases and questionable practices in the publication and dissemination of (communication) research, management of conflicts of interest, recording of studies and other monitoring methods to reduce bias and inadequate practices. Reproducibility Obstacles to practices of shared data and methods, replication studies; repli- (verification) cability and reproducibility of the published research and methods to improve them, effectiveness of the correction and self-correction of scientific literature and techniques to improve them. Evaluation Effectiveness, cost and benefits of new and old methods of peer reviewing and other scientific assessment approaches and methods to improve them. Incentives Precision, efficacy, costs and benefits of old and new approaches to evaluate (recognition) and order the performance, quality and value of the research of individuals, teams and institutions.
Source: Adapted from Ioannidis, Fanelli, Dunne & Goodman (2015)
Biases and questionable research practices
The past few years have witnessed a growing body of literature about the biases and errors that researchers make during the collection, analysis, storage and dissemination of data and results (for a general overview with emphasis on psychology research see Sijtsma, 2016 and related articles); as a result, there is currently a reasonable amount of evidence for the high prevalence of some questionable research practices (e.g. for the area of cognitive sciences see Ioannidis et al., 2014). Graph I shows an ideal version of the hypothetical-deductive model of the scientific method. Each stage can be associated with potential risks, which, taken together can help to undermine the robustness of the published research and can affect the capacity for self-correction of the science. Some of the phenomena receiving the most attention owing to their high prevalence include (Munafò et al., 2017):
■ P-hacking, also known as selective reporting. This questionable practice has been central to the debate on the statistical crisis of
204 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
the science. This occurs when researchers try out different statistical analyses or different specifications for the eligibility of data in an attempt to find statistically significant results. When they obtain these in some of the numerous settings they have explored, only the successful ones are reported. Some of the most frequent practices identified in this group include: − Omitting experimental conditions that produce data inconsistent with expected results and only analyzing and reporting results of the subgroup of remaining conditions; combining or separating treatments a posteriori, after obtaining the results; not reporting all the variables studied but only those associated with the desired results; defining a dependent variable a posteriori, for example, only using some items after proving significant effects in this redefined operative measure, and not reporting this. − Omitting or not omitting outliers or other observations in relation to re-assessments of the null hypothesis; or including or excluding covariables in view of the results obtained and not reporting this. − To perform the analysis while collecting experimental or study data and to stop compiling data when a statistically significant result is obtained. In other words, to increase the size of a sample stepwise, assessing the statistical interest at each step, until reaching statistical significance and then not collecting any more data. ■ HARKing, a term that refers to formulation of the research hypothesis a posteriori: after obtaining the result. The most common case is to report an exploratory study as being confirmatory. For example, to not report the respecifications of the original model that are later included in the final model. ■ Publication biases. The researchers tend to not publish studies in which they have not obtained statistically significant results, this is closely linked to the environment and culture of contemporary scientific publication.
The question, after having established the existence of threats to a reliable and robust accumulation of scientific evidence, is how to overcome these. There are multiple possible solutions that have been proposed, varied in nature and situated at different levels in the meta- research diagram shown in Table I. In the following sections we focus
Revista de Educación, 381. July-September 2018, pp. 199-221 205 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
on three of these; although they do not cover the whole range they are inclusive and are very present in the literature as useful lines of action.
GRAPH I. Threats to reproducibility in the scientific research process.
Publication Publish Generate Failuretocontrol biases and/or carry and specify bias out the study hypothesis
HARKing
P-hacking Interpret Design the Lowstatistical the results study power potencia estadística
Analyze the Conduct the data and Insufficient quality P-hacking study,collect study the the data control hypothesis
Source: Munafò et al. (2017)
Alternative and “new” emphasis on the statistical analysis of scientific data
The list of questionable research practices mainly includes actions that inflate the type I error, in other words, that increase the false-positives, so their analysis is strongly associated with the almost seventy year controversy about null hypothesis testing (Balluerka, Gómez et al., 2005; Fernández-Cano & Fernández-Guerrero, 2009; Harlow, Mulaik & Steiger, 2016). Although this can be considered as a highly familiar area, this debate has returned with renewed vigor within the context of the reproducibility crisis.
206 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
It has been frequently reported that the ecosystems of scientific production and publication gravitate almost exclusively toward a research model based on null hypothesis testing. This creates a strong incentive to obtain statistically significant results, which are the only results published, which in turn directly link to publication bias and selective reporting. In fact, some authors have even directly blamed the reproducibility crisis on the use of significance tests, and openly and simply defend the idea of not using them. One noteworthy representative of this position is Cummings, an advocate of the so-called new statistics (Cumming, 2014), who promotes the exclusive use of estimation instead of hypothesis testing, identifying three key elements: the use of effect sizes, confidence intervals and meta-analysis. The most extreme positions have had some impact. The most noteworthy example is that of a reasonably mainstream psychology journal, indexed in JCR (Basic and Applied Social Psychology), which in 2015 expressly demanded that null hypotheses tests not be used in its original articles (Trafimow & Marks, 2015). There has also been an evident resurfacing of the Bayesian approach to statistical inference (Mulder & Wagenmakers, 2016). But these somewhat more extreme approaches have also been widely contested from more moderate, and in our opinion more carefully considered, standpoints, which suggest that the correct use of hypothesis testing and its careful interpretation are crucial to scientific data analysis (Morey, Rouder, Verhagen & Wagenmakers, 2014). In fact, exclusively focusing on this aspect has been described as a red herring in the context of the crisis, as if doing away with the null hypothesis test would solve all the problems, and that using alternative approaches instead would guarantee the validity, robustness and effective advancement of the science (Savalei & Dunn, 2015). In this context, one can understand the publication of a recent institutional statement made by the American Statistical Association about the use of statistical significance and p-values in scientific research (Wasserstein & Lazar, 2015). In fact, as the text itself implies, there is nothing new, but the need for a formal announcement of this sort by the ASA clearly reflects the current state of the debate in the scientific setting. In any case, this renewed debate has resulted in a broad consensus about the need to review some practices and adopt others. The latter include the need to always report the effect size and its precision, by recording the corresponding confidence interval, which has been the recommended practice for more than three decades. Also, consolidation
Revista de Educación, 381. July-September 2018, pp. 199-221 207 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
of a net meta-analytical reasoning when considering the evidence available for a given problem, recognizing the limited information than can be obtained from any single study. The matter of effect size is certainly not a new one. Recommendations about this go back to the Eighties (Thompson, 2008), were widely disseminated in the 90’s (Wilkinson & The Task Force on Statistical Inference, 1999), have been present in all their publication manuals since 2001 (e.g. APA, 2001; APA, 2010) and also in the standards for the publication of empirical research of the AERA (AERA, 2006). But it is indeed being treated as if it were new. It would be interesting to find out the extent to which these recommendations have been adopted. Perhaps this explains why an old matter is still to some extent also a current one. The work of Peng, Chen, Chiang & Chiang (2013) is interesting in the North American context. The authors compile and analyze 31 previous reviews that had focused on assessing the use and reporting the effect size in North American journals of Psychology and Education. To this detailed and enormous review of reviews they add their own review of 451 articles published in 2009 and 2010. The group of 32 reviews is separated into two periods: before and after 1999 (the time of publication of the renowned Wilkinson report and the TFSI). A total of 116 journals were reviewed, including many of those associated with APA and AERA. In summary, this work can be said to show two things. Firstly, an overall increase in the use of effect size indices was identified, understood as an increase in the mean rate of their presence in research articles, although this is still only modest. In spite of a great variability among the journals, the overall rates can give a general idea. Before 1999, the mean rate was around 29.9% (median 29.4%) and after 1999 it was around 54.7% (median 58%). Secondly, incorrect practices in their estimation and interpretation were reported, which included a limited use of confidence intervals for effect sizes. It is evident therefore that adoption of the recommendations is slow and difficult. In the area of Spanish educational research we did not find any published studies on the use of effect size measurements and their confidence intervals. In Psychology, however, there were two rough indicators of the state of play. García, Campos and De la Fuente (2011) published reviews of 787 articles in four Spanish journals between 2003 and 2008 (Psicothema, Spanish Journal of Psychology, Psicológica, Internacional Journal of
208 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
Clinical and Health Psychology). They found a percentage of reporting of 21%, 32%, 3% and 19%, respectively, in each of the cited journals. As a final result, taking into account all the publications, they obtained a percentage of application of 21%. Caperos and Pardo (2013) analyzed the articles published in 2011 in four Spanish journals of Psychology indexed in the JCR database (Anales de Psicología, Psicológica, Psicothema, and Spanish Journal of Psychology). Their results show that 41% of the articles included some measure of effect size. It is noteworthy that these two reviews did not include the use of confidence intervals for effect size, a recommendation which was already present in the 2010 issue of the APA regulations and whose adoption will probably take a complementary or additional time. There clearly appears to be much room for improvement. Moreover, we would not expect to find rates much higher than this in educational research. The prevalence of statistical bias, malpractice and the very slow adoption of statistical innovations and improved practices can undoubtedly be used to discredit the validity of science. However, we believe that progress in the use of rigorous statistical methods and their careful interpretation are essential, powerful and solid characteristics of good science and key to upholding its integrity.
A renewed boost to replication and reproducibility: the case of sciences of education and behavior
In the area of Social Sciences replication studies are uncommon in spite of the key contribution they make to the advancement of scientific knowledge (Makel & Plucker, 2014). In fact, until just a few years ago there had been no systematic review of replication in educational research. In the work of Makel & Plucker (2014), cited previously, for the 100 journals included in the category Education & Educational Research of the JCR with an impact factor of over 5 years (issue 2011) all articles which included the term replicat* in the text were analyzed, taking into account the complete history of each publication. The overall rate of replication found in the study was 0.13% (221/164,589 articles), eight times less than that estimated in Psychology by the same authors (Makel, Plucker & Hegarty, 2012). A positive evolution of this over time was also observed. Since 1990 the rate has been almost four
Revista de Educación, 381. July-September 2018, pp. 199-221 209 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
times higher than this. In other words, replication studies have increased from around 1 in every 2000 articles to almost 1 in every 500. Regarding the analysis of results, the general rate of successful replications was around 67.4%. Although these results, which suggest that most studies were successfully replicated, differed from those recorded for Medicine or the Health Sciences (being significantly less promising), they appear to be similar to rates recorded in Psychology. Nonetheless, the authors call to mind the fact that almost half of the replications (48.2%) were carried out by the same teams that had published the original study. Moreover, the replication rate was higher when performed by the same team or when there was some degree of overlap in the authorship than when conducted by a totally independent team. It is important, therefore, to consider possible limitations associated with self-replications. It is uncertain whether the higher success rates arise from greater knowledge and experience or if the biases and questionable research practices recorded in the published literature of the Social Sciences are also replicated, a question that requires to further investigation. In any case, it appears that replication is rare and should be encouraged. As the authors of this work explain, replication is not the panacea and will not resolve all the challenges and problems related to the rigor, reliability, precision and validity of educational research. However, to continue to ignore it, either implicitly or explicitly, suggests a deep lack of understanding about science and how to further its advancement. These considerations have extended throughout the area of Psychology, where the crisis we describe here has been profound and has specifically developed in the past few years in the phenomena of replication and reproducibility. We briefly mention here the previously cited work of the Open Science Collaboration, published in Science. This article had, and still has, an enormous impact (1225 citations in Google Scholar by the 1st September 2017), and has promoted an interesting academic debate attracting both criticism and counter-criticism that has turned this study into an essential reference work in barely two years. It is an ambitious, collaborative and large-scale project in which independent research teams from all over the world participate to obtain an initial estimate of the reproducibility of Psychology research. More specifically, replications were performed of 100 correlational and experimental studies published in 2008 in three high impact Psychology journals (Psychological Science, Social Psychology, Experimental
210 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
Psychology: learning, memory and cognition) using high power designs and original materials where available. A total of 5 indicators were used to evaluate the success of a replication: significance and p-value, effect sizes, subjective assessment of the replication teams and meta-analysis of the effect sizes. The results of these five indicators are complex and difficult to summarize in a few paragraphs as they also include the analysis of possible factors related to reproducibility and exploratory studies were carried out with the aim of inspiring future research. However, some of the data are highly illustrative. A total of 97% of the original studies reported significant results, while results were only significant in37% of the replications. The mean effect size estimated in the replications was around 0.197, half of that reported in the original studies (0.403), corresponding to a substantial decrease. Finally, 39% of the effects were considered subjectively to be replications by the research teams. Taken together, the authors have drawn clear conclusions from these and the remaining results. A large proportion of the replications show weak support for the original results in spite of using the same material as that provided by the authors, reviewing beforehand the methodological reliability and the high statistical power to detect the original effect sizes. In any case, as the authors point out, this work leaves many questions unanswered and should be considered as a first step and not as the point of arrival. The original results offer tentative evidence, and the replications add additional confirmatory evidence. In some cases, the replications increase confidence in the original results. In other cases, however, they suggest that more research is required to establish the validity of the original results.
Open Science
Unanimous agreement about the need to increase transparency and openness in the processes used to carry out, disseminate and assess scientific research (as key factors to guarantee reproducibility) has led to the design and implementation in recent years of a range of initiatives, which together have given rise to what is known as Open Science (Morey et al., 2016; Nosek et al., 2015). An overall view and a good illustration of the kind of initiatives that are emerging are those promoted by the Center for Open Science, a North American non-
Revista de Educación, 381. July-September 2018, pp. 199-221 211 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
profit organization aimed specifically at developing free of charge and freely accessible technological tools for the promotion of open science and reproducibility (https://cos.io/). Here, we describe three of its projects that are particularly representative of developments in this field.
Guidelines for Transparency and Openness Promotion- TOP Guidelines
This proposal includes standards for eight modules that can be either partially or totally adopted and developed by scientific journals. Each module also includes three stringency levels for transparency. A flexible framework is defined that acknowledges that not all the standards are applicable to all journals or disciplines. The modules correspond to:
■ Citation standards ■ Data transparency ■ Transparency of analytical methods (code) ■ Transparency of research materials ■ Transparency of analysis and design ■ Pre-registration of studies ■ Pre-registration of analytical plans ■ Replication
An increasing number of journals are adopting this framework in a variety of disciplinary fields. For example, the journal Nature and those published by Springer and Biomed Central joined on 15th March 2017. We regard the concept of preregistered articles and reports to be particularly interesting. This is a new publication format that emphasizes the importance of the research question and the quality of the methodology by peer reviews prior to data collection (Nosek & Lakens, 2014). This entails provisionally accepting high quality research protocols that will be published regardless of the results obtained if the authors follow the specified methodology (see graph II).
212 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
GRAPH II. Preregistration of articles and analysis plans
Collect and Develop the Design the Draft the Publish the analyze the idea study report report data
Phase I Phase II Peer review Peer review
Source: Center for Open Science (https://cos.io/rr/)
The format is designed to encourage good practices and to do away with a series of questionable research practices, such as low statistical power, selective reporting or publication bias, while it enables flexibility to complete the report with unpredicted results, qualifying them as such. Emphasis is placed on the distinction between exploratory and confirmatory research to guarantee a correct assessment and evaluation of the results of the scientific research. At the time of writing this article, more than one hundred journals use this format, either including it as a standard part of their acceptance policy for original manuscripts, or as part of special editions.
Open Science Framework - OSF
Open science framework is an open and free of charge web environment that enables one to connect with and support the work flow of the research over the entire duration of the project, which can be used to collaborate, document, store, share and record research projects, materials and data. Other applications and platforms exist, but this is a good example of the types of tools and innovative software developed to support new practices of open science. Some of the journals that recommend using this platform as a repository to share data and materials and to preregister articles are: Biomed Central journals, Cognition, Nature, Psychological Science, Perspectives on Psychological Science, PloS, Science, Scientific Data.
Revista de Educación, 381. July-September 2018, pp. 199-221 213 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
Open Science Badges
These are used to expressly identify and recognize articles that comply with certain standards of transparency and openness. They can be granted after a simple declaration made by the authors or as the result of a peer review process, depending on the editorial policy of each journal. On writing this article around twenty journals had adopted these badges. For instance, requirements to be awarded an open data badge include:
■ Providing a URL, DOI, or another permanent path for accessing the data in a public open-access repository. ■ To provide together with the data sufficient information for an independent researcher to be able to reproduce the results reported in the article (metadata). ■ To have an open type license that allows others to copy, distribute and use the data, while maintaining the owners’ rights and credit where necessary (Creative Commons has recently defined several licenses of this type).
On the 1st August 2017 the American Psychological Association – APA announced an agreement with the Center for Open Science by which its journals offer open science badges and will use the Open Science Framework (OSF) as a data repository and to manage the preprint services (APA, 2017).
Conclusions: some challenges for the future of Spanish educational research
From the review conducted in the previous sections it is important to attempt to define some challenges that could be faced and must be overcome by educational research in our country. Here, we separate these into four different dimensions.
214 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
An analysis of current practices
When preparing this study we found there to be no strong tradition in reviewing and assessing educational research practices in Spain, unlike in other contexts, or in ours but in different areas. For example, in the area of Psychology studies can be found that analyze the use of statistical methods to identify good and bad practices and to formulate usage recommendations (in addition to the cited works of Caperos & Pardo, 2013 and Garcia, Campos & de la Fuente (2011); also see that of Izquierdo, Olea and Abad, 2014, among others). We can perhaps identify here a first question that must be addressed. We do not currently have sufficient information or evidence to enable us to diagnose, and where necessary to correct, incorrect practices. Therefore, a necessary and timely line of work would be to analyze the current statistical and methodological practices in Spanish educational research.
Training, informing and increasing awareness of researchers
Given the high prevalence with which questionable research practices have been identified, it seems reasonable to ask ourselves if, in fact, the researchers are always fully aware of engaging in them. Considering this possibility would perhaps help us to understand the situation better. Some of them may underestimate or be unaware of the complexity of some of the statistical techniques, for example, they may have difficulty understanding the impact of questionable research practices on type I error rates. We must focus on intentionality to distinguish malpractice (from a lack of information or insufficient training) from fraud. As explained in the literature (Sijtsma, 2016), there may be a need to change the initial and on-going training processes of researchers in statistics and methodology. Now is perhaps the moment to reflect upon initial postgraduate training and also upon the opportunity to arbitrate more permanent training approaches for active researchers. Moreover, the difficult and slow adoption of innovations and new good practices by researchers is receiving an increasing amount of attention (Henson, Hull & Williams, 2010; Sharpe, 2013; Cohen, 2017) and we believe that this issue should also be addressed in our setting.
Revista de Educación, 381. July-September 2018, pp. 199-221 215 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
Editorial policies: the role of editors and reviewers
At the end of the day, researchers want to publish their work and should be willing to comply with the demands of scientific journals. Initially, therefore, the journals play an essential role in the way in which research is conducted and disseminated. It would also be reasonable to deduce that if the journals explicitly impose adequate statistical and methodological standards for the acceptance of original manuscripts, which are accepted and considered as a standard of reference by the reviewers, and do not penalize or invite the publication of replications and encourage open science policies then this would have a direct positive effect, as appears to be occurring in other settings (see Munafò et al., 2017). In practice, we consider this to be a fantastic challenge that has arisen in our area and setting. During this decade in our country we have presided over an extraordinary consolidation of a large number of quality scientific journals in Education, witnessing a great and often selfless effort by part of the research community, and especially of editors and reviewers (Ruiz-Corbella, Galán & Diestro, 2014). A large number of publications have entered international indices and we personally consider that just to maintain previous levels of achievement would already be a challenge in itself. Nonetheless, we believe that these lines of action point towards a necessary way forwards for the future, a journey already embarked upon by some prestigious journals of education (E.g. see Lopez, Valenzuela, Nussbaum & Tsai, 2015).
Academic and scientific policies: the role of agencies funding and assessing research production
Last, but undoubtedly not least, we believe it is essential to consider the key role of the agencies that finance and assess the scientific production of researchers, teams and institutions, because, ultimately, they are the ones that design the incentive and recognition systems that greatly modulate individual and group behavior. In the assessment of production if quantity is awarded above all else and the ecosystem in which researchers work is mainly dominated by the pressure to publish, then it will be difficult for some of the practices described here to improve. If openly publishing data is not actively encouraged in any way then it is unlikely to become
216 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade more widespread. Moreover, if participation in collaborative broader scope projects and research consortia are not encouraged this will hinder their introduction and consolidation in practice. We agree, therefore, with authors who have emphasized the importance of introducing changes at this institutional level (Smaldino & McElreath, 2016), which is where most of the challenges described throughout this article are concentrated.
One final reflection
We essentially aim to emphasize in this work that scientific progress is an accumulative process to reduce uncertainty (rather than to obtain the right answers), and that this process can only be successful if the science itself is based on a natural and systematic scepticism about its own findings. We consider that the analysis carried out clearly shows that there is room for improvement in scientific practices. However, we hope we have also shown that, faced with the very plausible hypothesis that the current environment of science and scientific culture can be negatively affecting the validity and reproducibility of its results, the scientific system is behaving as it should do: systematically and critically reviewing its own practices and designing new methods of self-correction. We consider that a perspective like the one proposed here could help us to reflect on our own practices and, ultimately, contribute to the advancement of educational research in our country.
References
American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6), 33–40. doi:10.3102/0013189X035006033. American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: American Psychological Association. American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC:
Revista de Educación, 381. July-September 2018, pp. 199-221 217 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
American Psychological Association. American Psychological Association. (2017, August 1). APA Journals Program collaborates with Center for Open Science to advance open science practices in psychological research. Retrieved from: http:// www.apa.org/news/press/releases/2017/08/open-science.aspx. Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), 452-454. doi:10.1038/533452a Balluerka, N., J. Gómez, et al. (2005). The controversy over null hypothesis significance testing revisited. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences 1(2): 55-70. Caperos, J.M., & Pardo, A. (2013). Consistency errors in p-values reported in Spanish psychology journals. Psicothema, 25(3), 408-414.doi: 10.7334/psicothema2012.207. Cohen, B. H. (2017). Why the resistance to statistical innovations? A comment on Sharpe (2013). Psychological Methods, 22(1), 204-210. doi: 10.1037/met0000058. Cumming, G. (2014).The new statistics: why and how. Psychological Science, 25,7–29.doi:10.1177/0956797613504966. Fernández-Cano, A., & Fernández-Guerrero, I. (2009). Crítica y alternativas a la significación estadística en el contraste de hipótesis. Madrid: La Muralla. García, J. Campos, E., & De la Fuente, L. (2011). The use of the effect size in JCR Spanish journals of Psychology: from theory to fact .The Spanish Journal of Psychology, 14(2), 1050-1055. Goodman, S.N. (2016). Aligning statistical and scientific reasoning. Misunderstanding and misuse of statistical significance impede science. Science, 352 (6290), 1180-1181. doi: 10.1126/science.aaf5406. Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.). (2016). What if there were no significance tests? Classic Edition. New York: Routledge. Henson, R.K., Hull, D.M., & Williams, C.S. (2010). Methodology in our education research culture: toward a stronger collective quantitative proficiency. Educational Researcher, 39(3), 229-240. doi: 10.3102/0013189X10365102. Ioannidis, J. P., Fanelli, D., Dunne, D. D., & Goodman, S. N. (2015). Meta-research: evaluation and improvement of research methods and practices. PLoS Biology, 13(10),1-7. doi: 10.1371/journal.pbio.1002264. Ioannidis, J. P., Munafo, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014). Publication and other reporting biases in cognitive sciences:
218 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
detection, prevalence, and prevention. Trends in cognitive sciences, 18(5), 235-241. Izquierdo, I., Olea, J., and Abad, F.J. (2014). Exploratory Factor Analysis in validation studies: uses and recommendations. Psicothema, 26(3), 395-400. Ledgerwood, A. (2014). Introduction to the Special Section on Advancing Our Methods and Practices Perspectives on Psychological Science, 9(3) 275–277. doi: 10.1177/1745691614529448. Lopez, X., Valenzuela, J., Nussbaum, M. & Tsai, C.(2015). Some recommendations for the reporting of quantitative studies (Editorial). Computers & Education, 91,106-110. Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43, 304 –316. doi: 10.3102/0013189X14545513. Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives in Psychological Science, 7, 537–542. doi:10.1177/1745691612460688. McNutt, M. (2014). Journals unite for reproducibility (Editorial). Science, 346(6210), 679-679. doi:10.1038/515007a. Morey, R. D., Rouder, J. N., Verhagen, J., & Wagenmakers, E. J. (2014). Why hypothesis tests are essential for psychological science: a comment on Cumming (2014). Psychological science, 25(6), 1289-1290. Morey, R.D. et al. (2016). The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review. Royal Society Open Science, 3, 150547. doi: http://dx.doi.org/10.1098/ rsos.150547. Mulder, J. & Wagenmakers, E.J. (2016). Editors’ introduction to the special issue ‘‘Bayes factors for testing hypotheses in psychological research: Practical relevance and new developments’’. Journal of Mathematical Psychology, 72, 1–5. Munafò, M. R., Nosek, B. A., Bishop, D. V., Button, K. S., Chambers, C. D., du Sert, N. P., ... & Ioannidis, J. P. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021. doi:10.1038/s41562-016- 0021. Nature (2014). Journals unite for reproducibility (Editorial). Nature, 515(7525), 7. Nosek, B. A., et al. (2015). Promoting an open research culture. Science, 348, 1422-1425. DOI: 10.1126/science.aab2374.
Revista de Educación, 381. July-September 2018, pp. 199-221 219 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
Nosek,B. A. & Lakens, D.(2014). Registered reports. A method to increase the credibility of published results. Social Psychology, 45, 137-141. doi: 10.1027/1864-9335/a000192. Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science, 349 (6251). doi: 10.1126/science. aac4716. Peng, C-Y., Chen, L-T., Chiang, H-M., & Chiang, Y-C. (2013). The impact of APA and AERA guidelines on effect size reporting. Educational Psychology Review, 25, 157-209. doi: 10.1007/s10648-013-9218-2. Peng, R.D. (2015). The reproducibility crisis in science. A statistical counterattack. Significance, 30-32 Perspectives on Psychological Science. (2012). Special section on replicability in psychological science: A crisis of confidence? Retrieved from: http://pps.sagepub.com/content/7/6.toc Perspectives on Psychological Science. (2014). Special section on Advancing our methods and practices. Retrieved from: http://journals. sagepub.com/toc/ppsa/9/3. Ruiz-Corbella, M., Galán, A. & Diestro, A. (2014). Las revistas científicas de Educación en España: evolución y perspectivas de futuro. RELIEVE, 20 (2), art. M1. doi: 10.7203/relieve.20.2.436. Savalei V., Dunn E. (2015). Is the call to abandon p-values the red herring of the replicability crisis? Frontiers in Psychology, 6, 245. doi: 10.3389/ fpsyg.2015.00245. Schweinsberg, M. et al. (2016). The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline. Journal of Experimental Social Psychology, 66, 55-67. doi: https://doi. org/10.1016/j.jesp.2015.10.001. Sharpe, D. (2013). Why the resistance to statistical innovations? Bridging the communication gap. Psychological Methods, 18(4), 572. doi: 10.1037/a0034177. Sijtsma, K. (2016). Playing with data–Or how to discourage questionable research practices and stimulate researchers to do things right. Psychometrika, 81(1), 1-15. doi:10.1007/s11336-015-9446-0. Sijtsma, K., Veldkamp, C.L.S. & Wicherts, J.M. (2016). Improving the conduct and reporting of statistical analysis in Psychology. Psychometrika, 81(1), 33-38. doi:10.1007/s11336-015-9444-2. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis
220 Revista de Educación, 381. July-September 2018, pp. 199-221 Received: 09-04-2017 Accepted: 23-02-2018 Blanco-Blanco, Á. State of scientific practices and educational research. Potential challenges for the next decade
allows presenting anything as significant. Psychological Science, 22, 1359–1366. doi:10.1177/0956797611417632. Smaldino, P.E., McElreath, R. (2016). The natural selection of bad science. Royal Society Open Science, 3, 160384. doi:http://dx.doi.org/10.1098/ rsos.160384. Thompson, B. (2008). Computing and interpreting effect sizes, confidence intervals, and confidence intervals for effect sizes (pp. 246-262). En Osborne, J.W. (2008)(Ed.). Best Practice for Quantitative Methods. London: Sage. Trafimow, D. & Marks (2015). Editorial. Basic and Applied Social Psychology, 37 (1), 1–2. Waldman, I. D., & Lilienfeld, S. O. (2015). Thinking about data, research methods, and statistical analyses: Commentary on Sijtsma’s (2014) “Playing with data”. Psychometrika, 81(1), 16-26. doi:10.1007/s11336- 015-9447-z. Warschauer, M., Duncan, G. J., & Eccles, J. S. (2015). Inaugural Editorial: what we mean by “open”. AERA Open,1 (1),1-2. doi: 10.1177/2332858415574841. Wasserstein, R. L., & Lazar, N. A. (2016). The ASA’s statement on p-values: context, process, and purpose. American Statistician, 70(2), 129-133. doi: 10.1080/00031305.2016.1154108. Wigboldus, D. H. J., & Dotch, R. (2015). Encourage playing with data and discourage questionable reporting practices. Psychometrika, 81(1), 27-32. doi:10.1007/s11336-015-9445-1. Wilkinson, L., & The Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594–604. doi:10.1037/0003-066X.54.8.594.
Información de contacto: Ángeles Blanco Blanco. Universidad Complutense de Madrid, Facultad de Educación, Departamento de Investigación y Psicología en Educación. Rector Royo Villanova s/n 28040 Madrid. E-mail: [email protected]
Revista de Educación, 381. July-September 2018, pp. 199-221 221 Received: 09-04-2017 Accepted: 23-02-2018
Social Capital and Social Networks of Teachers: Systematic Review1
Capital Social y Redes Sociales de Maestros: Revisión Sistemática
DOI: 10.4438/1988-592X-RE-2017-381-387
Susana López Solé Mireia Civís Zaragoza Jordi Díaz-Gibson Universidad Ramón Llull
Abstract Recent studies demonstrate the importance and applicability of the concept of social capital in education, especially in relation to teachers, focusing on the analysis of social structures to understand how relationships between them can facilitate or inhibit access to different resources. With this purpose in mind, the present study performs a systematic review of the research that has been carried out on an international level in relation to teachers’ social capital and social networks to understand and demonstrate the importance of these constructs in the improvement of the educational praxis, as well as to promote their research. A search in the ERIC and Web of Science databases during the period 2000- 2015 reports 143 studies that meet the defined inclusion criteria and, of these, 14 studies between the years 2008-2015 that focus on the figure of the teacher, which this study analyses in greater depth. Most teacher-focused studies explore the improvement of teacher support and professional development through interactions and the creation of professional learning communities. The results propose a classification model of the 14 studies in relation to social capital.
(1) This work was supported by Fons Socials Europeus and Secretaria d’Universitats i Recerca del Departament d’Economia I Coneixement de la Generalitat de Catalunya (2015 FI_B 00948, 2016 FI_B1 00162, 2017 FI_B2 00142).
Revista de Educación, 381. July-September 2018, pp. 223-247 223 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Specifically, it is classified according to its conceptualization, based onits different structural, relational and cognitive dimensions, and its measurement, taking into account resources and positions in the network. However, results point to the need for greater consensus, due to the heterogeneity and variability of the studies reviewed. Finally, in the studies there is an agreement to explore the social capital of teachers from the analysis of social networks in order to analyse both formal networks, from the intentional support provided by schools, and informal networks, which appear spontaneously from personal relationships.
Key words: social capital, social networks, education, teachers, systematic review
Resumen Estudios recientes demuestran la importancia y la aplicabilidad del concepto del capital social en educación, con especial interés en los maestros, centrándose en el análisis de las estructuras sociales para comprender cómo las relaciones entre ellos pueden facilitar o impedir el acceso a diferentes recursos. Con este propósito, el presente estudio realiza una revisión sistemática de las investigaciones que se han realizado a nivel internacional en relación al capital social y a las redes sociales de maestros para conocer y demostrar la importancia de estos constructos en la mejora de la praxis educativa, así como para fomentar su investigación. La búsqueda en las bases de datos Eric y Web of Science durante el periodo 2000-2015, reporta 143 estudios que cumplen con los criterios de inclusión definidos y, dentro de éstos, 14 trabajos entre los años 2008-2015 que pertenecen a la figura de los maestros, en los que se profundiza en este estudio. La mayoría de los estudios de maestros exploran la mejora del apoyo y del desarrollo profesional, a través de las interacciones y la creación de comunidades profesionales de aprendizaje. En los resultados se propone un modelo de clasificación de los 14 estudios en relación al capital social. Concretamente, se clasifica según su conceptualización, en base a sus diferentes dimensiones estructurales, relacionales y cognitivas, y su medición, teniendo en cuenta los recursos y las posiciones en la red. Sin embargo, los resultados apuntan a la necesidad de mayor consenso, debido a la heterogeneidad y variabilidad de los estudios revisados. Finalmente, en los estudios existe un acuerdo para explorar el capital social de maestros a partir del análisis de redes sociales con el fin de analizar tanto las redes formales, a partir del soporte intencionado proporcionado por las escuelas, como informales, que aparecen de manera espontánea de las relaciones personales.
Palabras clave: capital social, redes sociales, educación, maestros, revisión sistemática
224 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Introduction
Today, we form part of a complex society characterised by constant, high- speed changes, which have become a daily reality in our lives (Bauman, 1999; Morin, 2004). In the last decades, we have experienced dramatic social changes in the areas of the economy, employment and the family, which have triggered a series of transformations in people’s socialisation process (Bolívar, 2006). Hence, interconnections have a fundamental role to play in our adapting to the changes that could lead to fragmentation and isolation, if we assume a systemic understanding of the world. The concern for the pedagogical innovation we are undergoing today responds largely to the search to offer a response to the changes our society is currently experiencing. In this regard, there is an increasing inclination towards the social side of education for dealing with today’s educational challenges, given the potential of connections and interactions, social capital being their ultimate expression. Faced with a growing diversity in the classroom and students’ environments (Clayton, 2007), teachers have a fundamental role in both student success and educational improvements (Piqué, Comas & Lorenzo, 2010; Vaillant & Manso, 2013). So, guaranteeing that teachers develop their work in this social and educational context becomes a strategic priority in education, which is why its mission is increasingly complex and there are more and more requirements regarding competencies and professionalism (Darling-Hammond, 2006; Schleicher, 2012).
Theory and Concept of Social Capital
Taking Lin’s definition (1999), social capital refers to the resources embedded in a social structure that are accessible and mobilised by an intentional action. In recent decades, social capital has emerged as a significant concept in the area of social sciences (Lin, 1999; Daly, 2010; Dika & Singh, 2012). Its origins can be traced back to the late 1980s with the theories of sociologists Bourdieu (1986) and Coleman (1988). Since then, a large number of theorists have written about the concept of social capital in its diverse forms and contexts, like Granovetter (1973), Putnam (1995), Portes (1998), Lin (1999), Fukuyama (2001) and Burt (2004). From these
Revista de Educación, 381. July-September 2018, pp. 223-247 225 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
different approaches, it can be ascertained that the main components that make up social capital can be summarised in: networks, trust, norms and sanctions (Baron, Field & Schuller, 2000; Halpern, 2005). Networks and trust are often the two key elements in defining the concept (Baron, Field & Schuller, 2000). The network makes references to the ties between the different interconnected nodes that make up the social structure (Castells, 1996; Scott, 2000). Trust has been studied by different authors (Coleman, 1988; Puntam, 1995; Fukuyama, 1995) as a facilitator of communication and social order, which can lead to reciprocity among the actors in the network. Likewise, norms and sanctions are often elements used to define social capital and attributed to the values, expectations and rewards that arise from relationships (Halpern, 2005). In turn, three functions of social capital can be distinguished depending on the characteristics of their ties: bonding, bridging and linking (Putnam, 2000; Woolcock, 2001; Halpern, 2005). The bonding function refers to those relationships that are more intense and closer between people, such as that of close friends or family. Bonding, therefore, strengthens people’s identities and the formation of more homogeneous groups (Halpern, 2005) and relates with the strong ties concept, which plays a key role in people’s support and emotional well-being (Granovetter, 1973). Bridging refers to those relationships that are more distant and not so close, such as friends of friends. This kind of social capital is also characterised by relationships between people with different origins and experiences (Miller, 2012). Therefore, these kinds of relationships provide access to new resources and opportunities which could not be available in the case of bonding relationships. Bridging social capital can be linked with the weak ties proposed by Granovetter (1973), as these relationships are greatly useful to people when it comes to accessing information, opportunities or work. Lastly, linking social capital describes the vertical relationships between people and institutions which have dissimilar positions and an unequal distribution of resources and power (Halpern, 2005), as would be the case of access to services thanks to connections with structures and institutions with high levels of power (Hawkins & Maurer, 2010).. Finally, social capital can be defined through different dimensions: structural, relational and cognitive (Nahapiet & Ghoshal, 1998). The structural dimension refers to the networks and patterns of connections between the different actors in the network (Nahapiet & Ghoshal, 1998).
226 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Those individuals that have more central positions in the social networks, therefore, will have greater access to resources, and the networks will be denser in which individuals are more connected. The relational part links with trust and the quality of ties, through the norms, values and expectations shared by individuals from the network (Moolenaar, 2010; Tschannen-Moran, 2014). In this dimension, trust plays an essential role, since this is a relational element which exists in social relationships to uphold mutual obligations and expectations (Liou & Daly, 2014). Finally, the cognitive part is based on the knowledge that is shared and exchanged thanks to communication between the different members that make up the network and which share a language and narratives in the network (Liou & Daly, 2014). Cognitive social capital, therefore, is formed by resources that provide shared representations and interpretations and a system of meanings between the members of the network, such as norms and values (Nahapiet & Ghoshal, 1998). In this regard, greater social capital of teachers would mean a greater performance of their teaching activity (Dika & Singh, 2002; Smethem, 2007; Liou et al., 2016), and, consequently, research would suggest the importance of teachers’ collaboration networks to increase their competencies and the success of their students (Penuel, Riel, Krause & Frank, 2009; Daly & Finnigan, 2010). At the end of the 1990s there was a surge in the use of “social capital” in scientific articles in a wide range of disciplines (Halpern, 2005). Today, it is increasingly studied as a subject, also in the field of education, but its definition is increasingly unclear, given the variations that exist depending on the approach adopted by each different author.
Social Network Theory and Analysis
From the definition of social capital, we see that it can be measured and captured through the resources present in social networks, where the different actors can access and approach these resources. A network consists of a set of actors (people, organisations, teams...) which are connected through a particular kind of tie (Kilduff & Tsai, 2003; Daly, 2010). In this regard, the interest in the theory and analysis of social networks has notably increased in the last 20 years (Moolenaar, 2010;
Revista de Educación, 381. July-September 2018, pp. 223-247 227 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Carolan, 2013). In the field of education, this analysis has enabled the study of a wide variety of phenomena, such as political reforms (Coburn & Russell, 2008; Penuel, et al., 2009), educational leadership (Moolenaar, Daly & Sleegers, 2010; Daly & Finnigan, 2011) or professional learning communities (Sleegers, den Brok, Verbiest, Moolenaar & Daly, 2013; Rienties & Kinchin, 2014). The analysis of teachers’ social networks allows for the exploration of different aspects related to their initial training (Liou et al., 2016; Civís, López & Díaz-Gibson, 2017) and collaboration between professionals to improve their professional development (Baumard & Starbuck, 2005; Penuel, Sun, Frank & Gallagher, 2012), focusing, among other things, on understanding how the relationships between actors from a network can facilitate or impede access to resources (Cross, Borgatti, & Parker, 2002; Borgatti & Foster, 2003). The resources that can be obtained may be of different natures, depending on whether they are instrumental or expressive networks (Lin, 1982; Ibarra, 1993). Instrumental networks refer to those networks that are related with academic work and development and which imply the exchange of information, knowledge, advice, etc. Expressive networks, however, are oriented towards transferring affective and emotional resources, such as friendship, for example. These expressive networks contain a higher level of trust when compared with instrumental networks, and tend to be more stable over time (Ibarra, 1993; Uzzi, 1997). Likewise, the analysis of social networks can be classified according to different levels of study: the egocentric or personal level of each actor, and the sociocentric level or view of the complete structure of the whole network (Borgatti, Everett & Johnson, 2013; Crossley et al., 2015; Hâncean, Molina & Lubbers, 2016). There are, therefore, three levels of analysis of social networks, depending on whether we are dealing with an analysis of the node (an actor), the dyad (two actors) or the network (all the actors from the network) (Daly, 2010; Borgatti, Everett & Johnson, 2013). With respect to measurements in these analyses, on a general level of the network are structural measurements like density, network size and global centralisation which offer us information about the network in terms of its cohesion and connectivity (Carolan, 2013; Borgatti, Everett & Johnson, 2013). At the level of the actor, there are different measurements that offer information about the position of a specific actor with respect to other actors from the network. The measurements of centrality (Freeman,
228 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
1979; Wasserman & Faust, 1998) are the ones that are most used in this analysis level, within which the most frequent are: degree centrality, betweenness centrality and closeness centrality. Degree centrality refers to the number of ties one actor has with other actors, drawing the distinction between the ties an actor sends out to others (out-degree) and the ties an actor receives from others (in-degree). For its part, betweenness centrality measures the degree to which actors control or mediate the relationship between pairs of actors that are not connected with each other. This centrality measure is related with the position of the broker or bridge, which plays a relevant role in the structure of the network, especially in the structural holes resulting from weak or absent ties in the connections between individuals (Burt, 2004). Lastly, closeness centrality measures the shortest distances from an actor to other actors.
Method
Objectives
The basic aim of the present study is to perform a systematic review of social capital and the social networks of teachers through international research on the subject carried out between 2000 and 2015. The study also has the following specific aims:
■ Understanding and demonstrating the importance of the constructs of social capital and social networks in the improvement of educational praxis, focusing on the figure of the teacher ■ Looking at the theory (and analysis) of social networks to understand the relevance of teachers’ social capital ■ Offering a proposal for conceptualising and measuring teachers’ social capital
Search Strategy
An exhaustive search through the literature was undertaken in the ERIC database, focused on the field of education (Marquès, 1999; Carolan,
Revista de Educación, 381. July-September 2018, pp. 223-247 229 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
2013), and in the main collection of the Web of Science database, focused on science (McVeigh & Bates, 2010). ERIC was chosen to provide access to a large number of bibliographic entries, journal articles and other materials related with educational research. Web of Science contains a large collection of bibliographic databases, citations and references from scientific publications from all knowledge areas, offering a transversal and multidisciplinary viewpoint. The period in which the articles were collected spanned from 01.07.15 to 12.11.15 and for the search, articles published between 2000 and 2015 were chosen using the keywords: “network analysis”, “social networks”, “social capital”, “educat*”, “school*” “and teach*”. The years 2000 and 2015 were chosen due to the growing trend in those years of publications in the area of education in relation to social networks and social capital (Carolan, 2013; Moolenaar, 2010). In the ERIC and Web of Science databases an advanced search was run which combined more than two descriptors (“DE” in ERIC) or subjects (“TS” in Web of Science). In both databases, boolean operators were used to search for entries that include all the terms separately (AND) and to search for entries that include any of the terms separately (OR). The result of the search was as follows: (DE “Network Analysis” OR DE “social networks”) AND DE “Social capital” AND (educat* OR school* OR teach*). While in the Web of Science database we ran the same search adapted to the database (“TS” instead of “DE”). The terms “Network Analysis” and “Social networks” allowed us to access literature based on networks and interactions. The term “Social capital”, in accordance with the literature, refers to resources obtained from networks. Lastly, the terms “educat”, “school”, and “teach” allowed us to set the context in the field of education. More specifically, these last terms allowed us to access diverse studies within the educational area (“educat”) which looked more deeply into the school or organisational sphere (“school”), as well as the figure of teachers and the teaching processes (“teach”).
Inclusion Criteria
The inclusion criteria established for the selection of articles, based on Liberati et al., (2009), were as follows:
230 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
■ Empirical studies with data collected through quantitative, qualitative or mixed methods. ■ Studies in the area of education and with the aim of improving the practice of education. ■ Studies that analyse and explore the concept of social capital based on the theory (and analysis) of social networks. ■ Studies published with a peer review system to ensure quality standards and reliability. ■ Studies published between 2000 and 2015.
Procedure
With the aim of illustrating the process, Figure 1 details the procedure followed and the different phases for performing the systematic review:
FIGURE I. Different phases of the systematic review
278 studies identified through database 237 studies identified through database searching (Web of searching (ERIC) Science)
481 studies after duplicates removed 34 duplicated studies, excluded
230 studies included on basis of title and abstarct 251 studies excluded on basis of title and abstract
143 studies included on basis of full -text 87 studies excluded on basis of full -text, with reasons
143 studies included in systematic review
14 teacher -focused studies
Source: Adapted from PRISMA (Liberati et al., 2009)
Once the duplicated studies in the databases were identified (34), we proceeded with the review of the 481 studies to evaluate their inclusion,
Revista de Educación, 381. July-September 2018, pp. 223-247 231 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
based on reading their titles and summaries. The evaluation of the studies was dichotomous and qualitative, marked as “accepted” or “rejected”, in accordance with the presence or absence of the defined criteria. In this phase, 251 studies were rejected for failing to fulfill the criteria. We then went on to the complete reading of the 230 remaining studies, to accept or reject them through an exhaustive analysis of the whole text. During this process, 87 studies were rejected, mainly due to the fact that they were not connected with the field of education and their main focus was not social capital for the improvement of the educational praxis. Finally, 143 studies were obtained that fulfilled the inclusion criteria for the systematic review, 14 of which were focused on teachers. Any disagreements during the procedure were resolved by consensus between five researchers. In addition, from each of the studies the following information was extracted in detail for its subsequent analysis and discussion:
■ Authors and years of publication ■ Geographical context of the study ■ Sample of the study (number of participants and description) ■ Main subject matter of the study ■ Approach to the theory of social capital and the theory (and analysis) of social networks ■ Main results and conclusions ■ Comments and observations
Results
The results obtained from the 14 teacher-focused studies mainly indicate social capital as a measure for supporting and furthering their professional development. As regards their approach to the theory of social networks, the studies differentiated between bonding social capital (or stronger, emotional ties) and bridging social capital (weaker, instrumental ties) in accordance with Putnam’s classification (2000), highlighting the importance of weak networks (weak ties) and strong networks (strong ties) (Granovetter, 1973). In these cases, the tie strength depends on the frequency and closeness of the social relationships. Likewise, the studies used levels of
232 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review analysis of social networks both on a sociocentric and egocentric level, with particular emphasis on measurements of degree and closeness centrality to calculate the social capital at the level of a single actor, as well as the measure of density across the whole network. The implications of most of these studies can be found in the methodology used for their analysis of social networks. The quantitative methods use social network questionnaires (sociocentric level) (Wasserman & Faust, 1998; Crossley et al., 2015), and tools such as the name and position generator (egocentric level) (Lin, 2001; Molina, 2005; Van Der Gaag & Snijders, 2005) for their subsequent analysis. The qualitative methods are based on visual mapping techniques, like concentric circles (Kahn & Antonucci, 1980) and conducting interviews and focus groups. Lastly, we also found studies that used mixed methods. Below is a proposal for classifying these studies in terms of their conceptualisation and measure of social capital, with the aim of contrasting the articles’ contributions and observing if there are any notable differences or patterns.
Conceptualisation of Social Capital
The conceptualisation of social capital has been based on the relationship between the components of networks, trust, norms and sanctions (Baron, Field & Schuller, 2000; Halpern, 2005) and the structural, relational and cognitive dimensions (Nahapiet & Ghoshal, 1998) (Table I). In these studies there is agreement regarding the conception of social capital, such as the resources that result from social relationships, which is why in most of them, social capital is explained through structural aspects, which affect position in the network and the amount of relationships. There are also studies that highlight the more relational aspect, attributed to trust, emphasising the quality of relationships. Lastly, we found studies that reflect the cognitive part, attributed mainly to shared norms, which take into account professional dialogue and identification with the group. However, most of these studies analyse social capital as a multidimensional concept in which the different dimensions are dealt with simultaneously to then obtain a more complete analysis of the object of study from different approaches.
Revista de Educación, 381. July-September 2018, pp. 223-247 233 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
TABLE I. Classification of the conceptualisation of social capital
STRUCTURAL RELATIONAL COGNITIVE SC SC SC Study Networks Trust Norms Sanctions Schiff, D., Herzog, L., Farley-Ripple, E., ✔ & Thum, L. (2015) Colomer, S.E. ✔ ✔ ✔ ✔ (2014) Daly, A., Moolena- ar, N., Der-Marti- ✔ ✔ rosian, C., & Liou, Y. (2014) Liou, Y. & Daly, A. ✔ ✔ ✔ (2014) Schreurs, B. & De ✔ Laat, M. (2014) Ranieri, M., Manca, ✔ ✔ S., & Fini, A. (2012) Baker-Doyle, K. & ✔ Yoon, S. (2011) Cannata, M. (2011) ✔ ✔ ✔ Van Emmerik, H., Jawahar, I.M., ✔ ✔ ✔ Schreurs, B., & de Cuyper, N. (2011) Anderson, L. (2010) ✔ ✔ Frank, K. A. (2009) ✔ Maier, A. & Youngs, ✔ ✔ ✔ P. (2009) Penuel, W., Riel, M., Krause, A., & ✔ ✔ ✔ Frank, K. (2009) Coburn, C. & Rus- ✔ ✔ sell, J. (2008)
Source: prepared by the author
234 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Measuring Social Capital
To measure social capital we adopted Lin’s proposal (2001) and its subsequent adaptation by Carolan (2013) in which it is understood as an asset in the network. Specifically, in Table II two types of asset are mentioned (embedded resources and location of the network) and their measuring methods (resources from the networks, resources from contacts, brokerage, closure and tie strength). Embedded resources are focused on the value of the resources from the networks and from contacts, which can be accessed by a given actor in the network. Resources from the network refer to aspects such as rank, quality, variety and composition of the networks (advice, information…), while in contacts they refer to the characteristics of the different actors from the networks as regards status, occupation and power. For its part, the position of the network refers to the structure of the network beyond the content. In this approach there are different dimensions, depending on whether it is mainly about brokerage or bridge, the notion of structural holes and centrality measures (highlighting betweenness centrality) (Burt, 1992). In the closure dimension reference is made to general aspects of the network and its cohesion, highlighting the measure of density. Finally, the tie strength dimension refers to the distinction between strong and weak networks (Granovetter, 1973), as regards intensity, interaction and reciprocity. Below are the results obtained according to the different studies, where it can be seen that embedded resources are dealt with in practically all the studies, both on networks and on contacts. The position of the network in reference to its structure is often dealt with to a lesser extent although in a way that is complementary to the content of the resources.
Revista de Educación, 381. July-September 2018, pp. 223-247 235 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
TABLE II. Classification of the measure of social capital
EMBEDDED RESOURCES NETWORK LOCATIONS Network Contact Tie Study Brokerage Closure resources resources strength Schiff, D., Herzog, L., Farley-Ripple, E., ✔ ✔ & Thum, L. (2015) Colomer, S.E. ✔ ✔ (2014) Daly, A., Moolena- ar, N., Der-Marti- ✔ ✔ ✔ ✔ rosian, C., & Liou, Y. (2014) Liou, Y. & Daly, A. ✔ ✔ ✔ (2014) Schreurs, B. & De ✔ ✔ ✔ Laat, M. (2014) Ranieri, M., Manca, ✔ ✔ ✔ S., & Fini, A. (2012) Baker-Doyle, K. & ✔ ✔ ✔ ✔ Yoon, S. (2011) Cannata, M. (2011) ✔ Van Emmerik, H., Jawahar, I.M., ✔ Schreurs, B., & de Cuyper, N. (2011) Anderson, L. ✔ ✔ (2010) Frank, K. A. (2009) ✔ ✔ Maier, A. & ✔ ✔ Youngs, P. (2009) Penuel, W., Riel, M., Krause, A., & ✔ ✔ ✔ ✔ Frank, K. (2009) Coburn, C. & ✔ Russell, J. (2008)
Source: prepared by the author
236 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Conclusions
In this systematic review, social capital appears as a concept that shows the importance of interactions and connections. What these studies have in common is that the attention of the different challenges is contemplated from a relational point of view and considering the role of social ties. This way, a social perspective is adopted focused on the potential of the social ties that emerge as a new proposal through investment in social relationships, as opposed to investments in skills or knowledge, which characterises human capital (Coleman, 1988). Therefore, this concept, whose origins lie in sociology (Bourdieu, 1986; Coleman, 1988), becomes a key element in education for understanding the complex phenomenon and thereby contributing to the improvement of the education system (Dika & Singh, 2002). To fulfill this aim, the analysis of social networks is presented as a tool (Wasserman & Faust, 1998; Scott, 2000; Lin, 2001), bearing in mind the multiple measuring and calculation options this technique offers. Likewise, we can see the networks that are more dense and the networks in which more investment needs to be made in social relationships to achieve greater cohesion. In turn, the isolated and key actors can be detected for resources to be more efficiently distributed and appropriated. The 14 teacher-focused studies that were analysed, which corresponded to the years 2008-2015, are mainly focused on the potential of interactions between teachers in formal and informal networks to mobilise resources and to achieve positions in the network that offer them benefits for improving their support network, professional development and the success of their students. In this direction, there are an increasing number of studies that aim to demonstrate the usefulness of focusing on social ties between teachers (Dika & Singh, 2002; Smethem, 2007; Penuel et al., 2009; Daly & Finnigan, 2010; Moolenaar et al., 2010), something which implies dedicating a part of the working day in educational centres to discussions, cooperative work, etc. It is therefore useful and beneficial to focus our attention on the current, emerging educational models for creating and sharing knowledge, such as co-teaching or team teaching. Likewise, reference is made to the concept of professional learning communities to highlight the importance of the collectivity and the group in professional environments. These communities share and critically reflect upon their practice with the aim of improving the teaching of
Revista de Educación, 381. July-September 2018, pp. 223-247 237 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
students and teachers and are consist of collaborative interaction between educators, characterised by different elements such as shared values and missions, collaboration and trust (Hord, 1997; Bolam, McMahon, Stoll, Thomas & Wallace, 2005). To promote these interactions, there needs to be consideration of the organisational and structural elements of the centres to provide suitable physical and temporal spaces for activities such as seminars, debates or training workshops. In these studies, trust is a basic element of social capital (Coleman, 1990; Putnam, 1993; Fukuyama, 1995), to be able to complement the more structural elements of the network in the educational sphere and as an essential element in schools (Bryk & Schneider, 2002; Tschannen-Moran, 2014). However, the role of trust in the conceptualisation and measure of social capital is not clearly defined in the literature, due to the theorists’ diversity of perspectives on the concept (Coleman, 1988; Fukuyama, 1995; Putnam, 1995; Baron, Field & Schuller, 2000; Halpern, 2005). More specific studies are therefore needed for exploring the contribution of trust in structural and cognitive elements, as well as the role it plays in the distribution of resources and the different positions in the network. In this review, the role of trust is often attributed separately to the conceptual definition of social capital, in its most relational dimension, and in reference to more instrumental networks such as that of asking for advice, help or knowledge, as a source of support for teachers. In this regard, the advice network is what is most used, beyond the use of more expressive networks (Lin, 1982; Ibarra, 1993). Bearing in mind the classification of social capital as a multidimensional concept (Nahapiet & Ghoshal, 1998), the results show a need for more studies that explore cognitive aspects when it comes to the shared values, meanings and knowledge that make up the culture and atmosphere of the school as necessary conditions for facilitating social interactions and promoting social capital. In this direction, creating a collaborative and supportive atmosphere among professionals is a fundamental element for increasing the social capital and improving the professional development of teachers (Bryk & Schneider, 2002; Cosner, 2009; Daly, Moolenaar, Der- Martirosian & Liou, 2014). To do this, professionals from educational centres sharing reflections should be encouraged, along with the setting of shared goals. Likewise, the structural dimension of networks and relational dimension of trust should be contemplated, since all these aspects together can contribute to improving the collaboration levels of
238 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review individuals in the network (Nahapiet & Ghoshal, 1998; Goddard, 2003; Liou & Daly, 2014). Through the measure of social capital, emphasis is placed on the importance of valuable resources in teachers’ relationships, whether through networks or contacts, showing the need for more studies that highlight the position of actors in the networks, cohesion and tie strength. With respect to the position of the network, centrality measures are a key element to consider for relating the positions of teachers in the school, or during their training at university, with their academic and professional development (Hommes et al., 2012; Liou et al., 2016). The measures of cohesion are also relevant for assessing how a denser and more connected network allows teachers to work collaboratively and improve their skills and encourage resource exchange (Daly, 2010; Civís et al., 2017). Finally, most of the teacher-focused studies deal with experienced teachers, while there is only one case that deals with future teachers. In the literature, there are studies that show the importance of adopting this social perspective in teacher training and in university programmes (Liou et al., 2016; Civís et al., 2017), indicating that a collaborative climate also promotes improvements in their professional development (Bryk & Schneider, 2002; Cosner, 2009; Daly et al., 2014). Only one case refers to new teachers, at the outset of their professional career, bearing in mind that it is at this time that they need most support from their colleagues to be able to develop professionally (Ewing & Manuel, 2005; Gasevic, Zouaq & Janzen, 2013). Various studies show that providing effective support to new teachers could contribute to ensuring they remain in the profession (Yuen, 2012; De Vries, Vand de Grift & Jansen, 2013). That is why it is especially important to understand teachers’ motivation and experience in their first years of the profession (Smethem, 2007) in order to put in place the right policies that will avoid them giving up on their professional careers. To sum up, this study reveals the status of research on teachers from a social and relational perspective. It shows how various studies have emerged on the profession that complement and offer a new perspective on studies that focus more on curriculum, assessment and planning (Nelson & Sassi, 2000; Yuen, 2012; Nilsson & Loughran, 2012). However, the sample of 14 studies is evidence to suggest that this line of work is still in its early stages.
Revista de Educación, 381. July-September 2018, pp. 223-247 239 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Furthermore, this systematic review enables us to observe how, despite the importance placed on working with social capital and social networks, there is no across-the-board agreement on how to analyse and explore teachers’ social capital, since different stances are adopted, through the use of different approaches for understanding and measuring the concept. Therefore, in this field these aspects need to be explored in greater detail to understand how social capital works and its usefulness and to reach a greater consensus. That is why efforts need to be aimed at understanding social capital’s potential in education in general and, more specifically, for teachers, to come to an agreement regarding its analysis and evaluation and be able to thereby continue to develop in this direction, both in the area of teachers’ initial training and in their ongoing training and professional practice. All research has a series of limitations. Firstly, following the search strategy, the studies of the literature that were reviewed are those found in the databases, which were selected for their relevance and international scope, bearing in mind that there may be other studies in the same area that are not contemplated for this study. In spite of this, the vast amount of studies included have been reviewed exhaustively, keeping in mind the difficulty of managing the immense supply of information and classifying it. Lastly, considering the possible development of the study, due to the newness of the subject matter and its exponential growth, in a near future it would be recommendable to follow up on and analyse the new studies that may contribute new and relevant findings in the area. Likewise, it would be useful to formalise a proposal for a new model to be able to analyse social capital in educational contexts, which incorporates the different approaches explored regarding its conceptualisation and measurement.
References
Anderson, L. (2010). Embedded, emboldened, and (net)working for change: Support-seeking and teacher agency in urban, high-needs schools. Harvard Education Review, 80(4), 541-572.
240 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Baker-Doyle, K. J., & Yoon, S. A. (2011). In search of practitioner-based social capital: A social network analysis tool for understanding and facilitating teacher collaboration in a US-based STEM professional development program. Professional Development in Education, 37(1), 75–93. Baron, S., Field, J., & Shuller T. (2000). Social Capital. Critical Perspectives. Nueva York: Oxford University Press. Bauman, Z. (1999). Modernidad líquida. Buenos Aires, Argentina: Fondo de Cultura Económica. Baumard, P., & Starbuck, W. H. (2005). Learning from failures: Why it may not happen. Long Range Planning, 38, 281–298 Bolam, R., McMahon, A., Stoll, L., Thomas, S., & Wallace, M. (2005). Creating and sustaining professional learning communities. London: General Teaching Council for England, Department for Education and Skills. Bolívar, A. (2006). Familia y escuela: dos mundos llamados a trabajar en común. Revista de Educación, 339, 119-146. Borgatti, S.P., Everett, M.G. & Johnson, J.C. (2013). Analyzing Social Networks. London: Sage Publications. Borgatti, S. P., & Foster, P. (2003). The network paradigm in organizational research: A review and typology. Journal of Management, 29(6), 991- 1013. Bourdieu, P. (1986). Handbook of theory and research for the sociology of education. New York: Greenwood Press. Bryk, A. S., & Schneider, B. (2002). Trust in schools: A core resource for improvement. New York, NY: Russell Sage Foundation. Burt, R. S. (1992). Structural holes. Cambridge. MA: Harvard University Press. Burt, R. S. (2004). Structural holes and good ideas. American Journal of Sociology, 110, 349-399. Cannata, M. (2011). The role of social networks in the teacher job search process. The Elementary School Journal, 111(3), 477-500. Castells, M. (1996). Rise of the network society. Cambridge, MA: Blackwell. Carolan, B. V. (2013). Social network analysis and education. Theory, methods, and applications. Thousand Oaks, CA: Sage Publications. Civís, M., López, S., & Díaz-Gibson, J. (2017). El capital social dels estudiants del darrer curs de Magisteri. Una aproximació a les seves xarxes socials en el marc de la facultat. REIRE Revista d’Innovació i Recerca en Educació, 10(1), 34-56.
Revista de Educación, 381. July-September 2018, pp. 223-247 241 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Clayton, C. D. (2007). Curriculum Making as Novice Professional Development: Practical Risk Taking as Learning in High-Stakes Times. Journal of Teacher Education, 58(3), 216–230. Coburn, C. E., & Russell, J. L. (2008). District policy and teachers’ social networks. Education Evaluation and Policy Analysis, 30(3), 203–235. Coleman, J. (1988). Social capital in the creation of human capital. American Journal of Sociology, 94, S95-S120. Coleman, J. S. (1990). Foundations of social theory. Cambridge, MA: Harvard University Press. Colomer, S. E. (2014). Latina Spanish High School Teachers’ Negotiation of Capital in New Latino Communities. Bilingual Research Journal, 37(3), 349-365. Cosner, S. (2009). Building organizational capacity through trust. Educational Administration Quarterly, 45(2), 248-291. Cross, R., Borgatti, S., & Parker, A. (2002). Making invisible work visible. California Management Review, 44(2), 25–46 Crossley, N., Bellotti, E., Edwards, G., Everett, M.G., Koskinen, J., & Tranmer, M. (2015). Social network analysis for ego-nets. London, UK: SAGE. Daly, A. J. (2010). Social network theory and educational change. Cambridge, MA: Harvard Education Press. Daly, A. J., & Finnigan, K. (2010). A bridge between worlds: understanding network structure to understand change strategy. Journal of Educational Change, 11(2), 111-138. Daly, A. J., & Finnigan, K. S. (2011). The ebb and flow of social network ties between district leaders under high-stakes accountability. American Educational Research Journal, 48(1), 39–79. Daly, A. J., Moolenaar, N., Der-Martirosian, C., & Liou, Y.-H. (2014). Accessing capital resources: Investigating the effects of teacher human and social capital on student achievement. Teachers College Record, 116(7), 1–42. Darling-Hammond, L. (2006). Constructing 21st-Century Teacher Education. Journal of Teacher Education, 57(3), 300-314. De Vries, S., Van de Grift, W. J., & Jansen, E. P. (2013). Teachers’ beliefs and continuing professional development. Journal of Educational Administration, 51(2), 213-231. Dika, S. & Singh K. (2002). Applications of Social Capital in Educational Literature: A Crytical Synthesis. Review of Educational Research, 72 (1), 31-60.
242 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Ewing, R., & Manuel, J. (2005). Retaining quality early career teachers in the profession. Change: Transformations In Education, 8(1), 1-16. Frank, K. A. (2009). Quasi-ties: Directing resources to members of a collective. American Behavioral Scientist, 52, 1613–1645. Freeman, L.C (1979). Centrality in Social Networks Conceptual Clarification.Social Networks, 1, 215-239. Fukuyama, F. (1995). Trust: The social virtues and the creation of prosperity. New York: Free Press. Fukuyama, F. (2001). Social Capital, civil society and development. Third World Quarterly, 22 (1), 7-20. Gasevic, D., Zouaq, A., & Janzen, R. (2013). “Choose your classmates, your GPA is at stake!”: The association of cross-class social ties and academic performance. American Behavioral Scientist, 57(10), 1460- 1479. Goddard, R. (2003). Relational networks, social trust, and norms: A social capital perspective on students’ chances of academic success. Educational Evaluation and Policy Analysis, 25(1), 59-74. Granovetter, M. S. (1973). The strength of weak ties. American Journal of Sociology, 78 (6), 1360-1380. Halpern, D. (2005). Social Capital. Cambridge: Polity Press. Hâncean, M-G., Molina, J.L., & Lubbers, M. (2016). Recent advancements, developments and applications of personal network analysis. International Review of Social Research, 6(4), 137-145. Hawkins, R. L., & Maurer, K. (2010). Bonding, bridging and linking: How social capital operated in New Orleans following Hurricane Katrina. British Journal of Social Work, 40(6), 1777–1793. Hommes, J., Rienties, B., Grave, W. D., Bos, G., Schuwirth, L., & Scherpbier, A. (2012). Visualising the invisible: A network approach to reveal the informal social side of student learning. Advances in Health Education, 17(5), 743-757. Hord, S. (1997). Professional learning communities: Communities of continuous inquiry and improvement. Austin, TX: Southwest Educational Development Laboratory. Ibarra, H. (1993). Personal networks of women and minorities in management: a conceptual framework. Academy of Management Review, 18(1) 58-67. Kahn, R. L., & Antonucci, T. C. (1980). Convoys over the life course: Attachment, roles and social support. En P. B. Baltes y O. G. Brim
Revista de Educación, 381. July-September 2018, pp. 223-247 243 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
(Eds.). Life-span development and behaviour (pp. 383–405). New York: Academic Press. Kilduff, M., & Tsai, W. (2003). Social Networks and Organizations. London: Sage Publications. Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Ioannidis, J.P.A., et al. (2009). The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Medicine, 6(7), e1000100. Lin, N. (1982) Social Resources and Instrumental Action. In: P. V. Marsden & N. Lin (Eds). Social Structure and Network Analysis (pp. 131-45). Beverly Hills, CA: Sage. Lin, N. (1999). Building a network theory of social capital. Connections, 22(1), 28–51 Lin, N. (2001). Social capital: A theory of social structure and action.(1st Ed.). New York: Cambridge University Press. Liou, Y.-H., & Daly, A. J. (2014). Closer to Learning. Social Networks, Trust, and Professional Communities. Journal of School Leadership Volume, 24, 753-795. Liou, Y.-H., Daly, A. J., Canrinus, E. T., Forbes, C. A., Moolenaar, N. M., Cornelissen, F., & Hsiao, J. (2016). Mapping the social side of pre-service teachers: connecting closeness, trust, and efficacy with performance. Teachers and Teaching, 1-23. Maier, A., & Youngs, P. (2009). Teacher preparation programs and teacher labor markets: How social capital may help explain teachers’ career choices. Journal of Teacher Education, 60(4), 393–407. Marquès, P. (1999). Criterios para la clasificación y evaluación de espacios web de interés educativo. Educar, 25, 95-111. McVeigh, M. E., & Bates, M. J. (2010).Citation Indexes and the Web of Science. Encyclopedia of Library and Information Science, 2. Miller, P. (2012). Community-Based Education and Social Capital in an Urban After-School Program. Education and Urban Society, 44(1), 35- 60. Molina, J.L. (2005). El estudio de las redes personales: contribuciones, métodos y perspectivas. Revista de Metodología de las Ciencias Sociales, (10), 71-105. Morin, E. (2004). Introducción al pensamiento complejo. Barcelona: Gedisa.
244 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Moolenaar, N. M. (2010). Ties with potential. Nature, antecedents, and consequences of social networks in school teams. Unpublished doctoral dissertation. University of Amsterdam, The Netherlands. Moolenaar, N., Daly, J., & Sleegers, P. (2010). Occupying the principal position: examining relationships between transformational leadership, social network position, and schools’ innovative climate. Educational Administration Quarterly, 46(5), 623-670. Nahapiet, J., & Ghoshal, S. (1998). Social capital, intellectual capital and the organizational advantage. Academy of Management Review, 23(2), 242-266. Nelson, B. S. & Sassi, A. (2000). Shifting approaches to supervision: The case of mathematics supervision. Educational Administration Quartely, 36(4), 553-584. Nilsson, P. & Loughran, J. (2012). Exploring the development of pre- service elementary teachers’ pedagogical content knowledge, Journal of Science Teacher Education, 23(7), 699-721. Penuel, W. R., Riel, M. R., Krause, A., & Frank, K. A. (2009) Analyzing teachers’ professional interactions in a school as social capital: A social network approach. Teachers College Record, 111(1), 124–163. Penuel, W. R., Sun, M., Frank, K. A., & Gallagher, H. A. (2012). Using Social Network Analysis to Study How Collegial Interactions Can Augment Teacher Learning from External Professional Development. American Journal of Education, 119(1), 103-136. Piqué, B., Comas, A., & Lorenzo, N. (2010). Estratègies de pràctica reflexiva en la formació inicial de mestres d’educació infantil. Barcelona: Graó. Portes, A. (1998). Social capital. Annual Review of Sociology, 24, 1-24. Putnam, R. D. (1993). The prosperous community: Social capital and public life. American Prospect, 4, 35-42. Putnam, R. D. (1995). Bowling alone: America’s declining social capital. Journal of Democracy, 6 (1), 65-78. Putnam, R. D. (2000). Bowling alone: The collapse and revival of American community. New York: Simon & Schuster. Ranieri, M., Manca, S., & Fini, A. (2012). Why (and how) do teachers engage in social networks? An exploratory study of professional use of Facebook and its implications for lifelong learning. British Journal of Educational Technology, 43(5), 754-769.
Revista de Educación, 381. July-September 2018, pp. 223-247 245 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Rienties, B., & Kinchin, I. M. (2014). Understanding (in)formal learning in an academic development programme: A social network perspective. Teaching and Teacher Education, 39, 123–135. Schiff, D., Herzog, L., Farley-Ripple, E., & Thum, L. (2015). Teacher networks in Philadelphia: Landscape, engagement, and value. Perspectives on Urban Education, 12 (1), 1-17. Schleicher, A. (2012). Preparing Teachers and Developing School Leaders for the 21st Century: Lessons from around the World. Paris: OECD Publishing. Schreurs, B., & De Laat, M. (2014). The Network Awareness Tool: A web 2.0 tool to visualize informal networked learning in organizations. Computers in Human Behavior, 37, 385–394. Scott, J. (2000). Social network analysis. (2nd ed.). London: Sage. Sleegers, P. J. C., Den Brok, P., Verbiest, E., Moolenaar, N. M., & Daly, A. J. (2013). Towards conceptual clarity: A multidimensional, multilevel model of professional learning communities in Dutch elementary schools. Elementary School Journal, 114(1), 118–137. Smethem, L. (2007). Retention and intention in teaching careers: will the new generation stay? Teachers and Teaching, 13 (5), 465–480. Tschannen-Moran, M. (2014). Trust Matters: Leadership for Successful Schools .(2nd ed.). San Francisco: Jossey-Bass. Uzzi, B. (1997). Social structure and competition in interfirm networks: the paradox of embeddedness. Administrative Science Quarterly, 42 (1). Vaillant, E. & Manso, J. (2013). Teacher education programs: learning for worlwide inspiring experiences. Journal of Supranational Policies of Education, (1), 95-115. Van Der Gaag, M., & Snijders, T. (2005). The resource generator: social capital quantification with concrete items. Social Network, 27(1), 1–29. Van Emmerik, H., Jawahar, I.M., Schreurs, B., & de Cuyper, N. (2011). Social capital, team efficacy and team potency: The mediating role of team learning behaviors. Career Development International, 16(1), 82-99. Wasserman, S., & Faust, K. (1998). Social network analysis: Methods and applications. New York: Cambridge University Press. Woolcock, M. (2001). The place of social capital in Understanding Social and Economic Outcomes. SUMA Canadian Journal of Policy Research, 2(10),11-17.
246 Revista de Educación, 381. July-September 2018, pp. 223-247 Received: 23-10-2017 Accepted: 09-02-2018 López Solé, S., Civís Zaragoza, M., Díaz-Gibson, J. Social Capital and Social Networks of Teachers: Systematic Review
Yuen, L. H. (2012). The impact of continuing professional development on a novice teacher. Teacher Development, 16(3), 387-398.
Contact address: Susana López Solé. Universidad Ramón Llull, Facultad de Psicología, Ciencias de la Educación y del Deporte, Educación, c/ del Císter, 34, 08022 Barcelona. E-mail: [email protected]
Revista de Educación, 381. July-September 2018, pp. 223-247 247 Received: 23-10-2017 Accepted: 09-02-2018
Inquiry-based Science Education. A systematic review of Spanish production
La enseñanza de las ciencias basada en indagación. Una revisión sistemática de la producción española
DOI: 10.4438/1988-592X-RE-2017-381-388
David Aguilera Morales Tobías Martín-Páez Víctor Valdivia-Rodríguez Ángela Ruiz-Delgado Leticia Williams-Pinto José Miguel Vílchez-González Francisco Javier Perales-Palacios Universidad de Granada
Abstract Due to the growing importance given to teaching based on inquiry both at the international and national level, this article aims to further work related to this line of research published between 2007 and 2017 in the Spanish area of Didactic of Experimental Sciences. We reviewed 1941 articles, including 55 criteria for the selection, all according to the guidelines of the PRISMA statement (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). The results show the most prolific authors, the evolution of scientific production, the educational stages considered, the types of studies, the designs and the educational implications of their application in the classrooms. In light of these, we can conclude that there are not many authors with an excellent production regarding this subject, because this research’s line is still incipient in Spain. The attention devoted to the different educational stages by research in Didactic of Experimental Sciences is not equitable, with the most affected being the stages of Infantile and Primary Education. There are more articles of a theoretical
Revista de Educación, 381. July-September 2018, pp. 249-274 249 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
nature than empirical ones, given the lack of experimental studies that prove this teaching approach is patent. However, the educational implications derived from Spanish publications are similar to the international ones. Finally, we have to advocate for an increase in the number of studies aimed at evaluating the benefits of teaching based on inquiry in the different educational stages, focusing on the stages of Pre-school Education and Primary Education.
Keywords: Science education; teaching based on inquiry; educational implications; Spanish journals; systematic review.
Resumen Ante la creciente relevancia otorgada a la enseñanza basada en indagación tanto en el ámbito internacional como nacional, este artículo pretende aunar aquellos trabajos referentes a esta línea de investigación publicados entre 2007-2017 en las revistas españolas del área de Didáctica de las Ciencias Experimentales. Se han revisado 1941 artículos, incluyéndose 55 de ellos tras la aplicación de los criterios de selección establecidos, todo ello según las directrices de la Declaración PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). Los resultados muestran cuáles son los autores más prolíficos, la evolución de la producción científica, las etapas educativas consideradas, los tipos de estudios, sus diseños y las implicaciones educativas de su aplicación en las aulas. A la luz de estos, podemos concluir que no abundan autores o autoras con una excelsa producción referente a esta temática, pues esta línea de investigación es aún incipiente en España. Además, la atención dedicada a las distintas etapas educativas por la investigación en Didáctica de las Ciencias Experimentales no es equitativa, siendo las grandes perjudicadas las etapas de Educación Infantil y Primaria. Existen más artículos de naturaleza teórica que empírica, dado que queda patente la falta de estudios experimentales que testen este enfoque de enseñanza. No obstante, las implicaciones educativas derivadas de las publicaciones españolas son similares a las internacionales. Finalmente, hemos de abogar por un incremento de los estudios dirigidos a evaluar los beneficios de la enseñanza basada en indagación en las diferentes etapas educativas, incidiendo en las etapas de Educación Infantil y Primaria.
Palabras clave: Enseñanza de las ciencias; enseñanza basada en indagación; implicaciones educativas; revistas españolas; revisión sistemática.
Introduction
Inquiry as strategy for the teaching and learning of the sciences is paving the way as a growing alternative in the face of the need to improve the
250 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production current form of teaching sciences in educational centres. Some of the objectives pursued with it are as follows:
(1) Improving the image of science and attitudes towards it. (2) Contributing to scientific literacy. (3) Incentivising scientific professions.
It has undoubtedly been influenced by its use being recommended on the part of the Rocard Report (2007) and, more recently, the publication of the European Commission report: “Science Education for Responsible Citizenship” (Hazelkorn, 2015), which focuses heavily on this recommendation. Thus, it indicates that “Education policies and systems should: …Support schools, teachers, teacher educators and students of all ages to adopt an inquiry approach to science education as part of the core framework of science education for all…” (p. 10). Despite the importance given in recent times to this line of research, there are few studies that offer summaries of this subject. Specifically, in the last decade, we found four papers that revised inquiry-based teaching in the international sphere (Minner, Levy & Century, 2010; Alfieri, Brooks, Aldrich & Tenenbaum, 2011; Furtak, Seidel, Iverson & Briggs, 2012; Lazonder & Harmsen, 2016), whereas at a national level, the work of Romero-Ariza (2017) is the only one available to us. Although the differences between the revision of Romero-Ariza (2017) and the aforementioned work are evident, given that the author carried out a narrative revision in which she comments on results obtained by other revision studies, in this case meta-analysis, which deal with the subject in question. To expand on the information in this respect, this article reviews the research published between 2007 and 2017 in Spanish Magazines specifically focused on Experimental Science Teaching (EST), from the following research questions:
(1) Who are the most prolific authors in this line of research? What level of collaboration exists? (2) How many articles deal with inquiry-based teaching? How have they evolved over time? In which countries have they been written? (3) What educational stages are being researched?
Revista de Educación, 381. July-September 2018, pp. 249-274 251 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
(4) What type of studies are published on this subject? What designs do they have? (5) What educational implications do they set forth in relation to inquiry-based teaching?
Inquiry and science teaching
The first thing that should be defined in any study on inquiry is in what sense it is spoken about. As pointed out by Couso (2014), the term “inquiry” is “surprisingly polysemic in educational literature”. Therefore, when speaking about inquiry we can make reference to (Barrow, 2006, quoted by Couso, 2014):
■ The ability to research scientifically, if we focus attention on the development of this cognitive capacity. ■ The nature of scientific research, if we focalise on students understanding scientific methodology. ■ Teaching-learning strategies whose objectives focus on students not only understanding scientific concepts, but also acquiring abilities “of” and “relating to” scientific inquiry.
The two initial meanings refer to content to teach, to scientific procedures in the first case and about science in the second; the third, in contrast, is a manner of carrying out inquiry. The latter case talks about IBSE, Inquiry-Based Science Education, or IBL, Inquiry-Based Learning. However, a prior conceptual clarification becomes necessary to delimit this strategy and differentiate it from others that originate from different spheres of social and health sciences, with some common elements such as project-based teaching or problem-based learning. In figure I we have attempted to facilitate such a distinction, presenting a number of definitions on inquiry below1:
■ “The intentional process of diagnosing problems, critiquing experiments, distinguishing alternatives, planning investigations, researching conjectures, searching for information, constructing
(1) The following paragraphs are adapted from Martínez-Aznar (2017).
252 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
models, debating with peers, and forming coherent arguments” (Linn, Davis y Bell, 2004, p. 4). ■ “Appreciating the diverse ways in which scientists conduct their work; understanding the power of observations; knowledge of and ability to ask testable questions, make hypotheses; use various forms of data to search for patterns, confirm or reject hypotheses; construct and defend a model or argument; consider alternate explanations, and gain an understanding of the tentativeness of science, including the human aspects of science, such as subjectivity and societal influences” (Crawford, 2007, p. 614).
FIGURE I. Conceptual map. Source: adapted from Perales & Ayerbe (2016).
IBSE, upon which we focus our attention from this point on, is usually proposed as an alternative to traditional teaching. Its main characteristics are (Couso, 2014):
Revista de Educación, 381. July-September 2018, pp. 249-274 253 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
■ I t is organised in research environments, normally of the practical type. ■ Students acquire a more active role than in traditional environments, and teachers acquire a more passive role (act as guides). ■ A great deal of importance is afforded to attitude and motivation. ■ Students, generally working in groups, ask questions and obtain data (or use available data). ■ Students enjoy a lot of autonomy and capacity for decision-making. ■ It is designed in stages that simulate scientific methodology.
Regarding the level of student autonomy, Windschitl (2003) differentiates between the proposals focused on the “confirmation of experiences” discussed beforehand in the classroom and those set out as “open inquiry” (students decide on what to investigate and how to do it), as well as tackling “structured inquiry” strategies (the teacher provides both the question and the procedure to resolve it) and “guided inquiry” (the teacher provides the question and the students decide how to resolve it), each of them having an influence on students’ *different scientific skills to a greater or lesser extent (Table I).
TABLE I. Inquiry models and skills performed by students. Source: Bevins & Price (2016).
Inquiry models Inquiry skills grouped into dimensions
1. 4. Explanations Scientifically 2. Priority to 3. Explanations 5. Communicate Level connected to orientated evidence from evidence and justify knowledge questions
Determine Examine other Look how to Formulate that which resources for formulate logical Formulate explanations 3. Open constitutes formulating arguments to questions after grouping evidence and explanations on communicate evidence seek it work explanations
Guided in the Directed Orientated Select Orientated process of towards areas towards questions so that they 2. Guided formulating and sources strategies for and ask new seek certain explanations of scientific developing ones information from evidence knowledge communication
254 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
Provided with Clarify or Receive examples of Given possible Given guidelines specify data and are how to use connections to use in 1. Structured questions asked to evidence to with scientific order to focus assigned to analyse them formulate knowledge communication them explanations
Precise Give Receive Receive connection Receive specific 0. Confirmation response to data and are explanations between steps and / verification questions explained on their knowledge procedures for activities provided by how to results indicated to communication others analyse them them
In the Spanish sphere we can find precedents to this line of research, specifically in the works of D. Gil and collaborators, and the IRES (Investigación y Renovación Escolar) network. In the first case, specifically focused on science teaching and learning, Gil (1990, 1993) commits to the model of problem solving by research, establishing an analogy between science research and the resolution of problems, where the novice researcher (student) works under the direction and supervision of the lead researcher (teacher), using the open problems’ effect (preferably qualitative). In the second case, the proposal has a more transverse focus, considering research as an integrating didactic principal (Cañal, 1987), which should encourage the scientific spirit and the skills specific to scientific methodology in the apprentice; this should impregnate the most characteristic elements of the school curriculum such as objectives, content, communication relationships, etc.
Inquiry in the classroom
In relation to its implementation in the classroom, the stages of an IBSE proposal are, in harmony with scientific research, the following (Martínez-Chico, 2013):
(1) Identify problems or questions of a scientific nature (whose responses can be confirmed or rejected through evidence). (2) Issue hypotheses as possible responses to the problem or question (justified explanations).
Revista de Educación, 381. July-September 2018, pp. 249-274 255 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
(3) Seek evidence that confirms or refutes the hypothesis (through experiments or searches for information). (4) Analyse and interpret results. (5) Extract conclusions and communicate them.
Although IBSE admits different interpretations (Romero-Ariza, 2017) that depend on the conception of inquiry itself and previous teaching experiences (Windschitl, 2003), which is a fact that makes research, and even debate on inquiry, difficult (Yeomans, 2011), all coincide on two aspects (Couso, 2014):
■ The authenticity of scientific methodology must be brought to the classroom. ■ Students must be involved and motivated.
All of this paints a scenario where the questions to take into account and carry out are complex and, therefore, difficult to tackle, with this situation postulating itself one of the reasons why inquiry-based strategies do not yet enjoy a significant presence in science classrooms, with others being maintained, such as: lack of teacher preparation (Forbes & David, 2010; Lucero, Valcke & Schelles, 2013), scarcity of resources, time limitations, extensive curriculum and parental pressure (Anderson, 1996) or the conception of inquiry itself, and the way of implementing it in the classroom (Cuevas, Lee, Hart & Deakort, 2005; McDonald & Butler Soger, 2008). In addition to this, there is the fact that the scientific knowledge of teachers in practice and training is generally fragmented, superficial and weak (Murphy et al., 2007) and, moreover, they display a certain rejection of science and low levels of confidence (Vázquez & Manassero, 2008). Thus, in particular, some limitations detected in trainee teachers for the implementation of inquiry activities are related to the choice of the question to research, the design and the execution of the investigation (Lucero et al., 2013; Crujeiras & Puig, 2014). Once we have established the concept understood in this paper as inquiry (IBSE), its main characteristics and some of the principle barriers to these strategies reaching the classroom, what would still remain to be studied is whether those that have been implemented have had the desired impact. Although there are authors who consider inquiry to be the best method for science teaching and learning (Bevins & Price, 2016), or that
256 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
“teaching strategies that actively engage students in the learning process through scientific investigations are more likely to increase conceptual understanding than are strategies that rely on more passive techniques” (Minner, Levy & Century, 2010, p. 493), there are also those who assert that this is not the case (Cobern et al., 2010). In fact, there are studies that negatively correlate methodologies of inquiry in the classroom and academic performance in PISA tests (Areepattamannil, 2012; McConney, Oliver, Woods-McConney, Schibeci & Maor, 2014, quoted by Romero- Ariza, 2017). In this regard, it can be pointed out that there are few cases to analyse, as has been described. Nevertheless, some results are worth mentioning. In a recent study that analyses meta-analyses measuring the extent of the effect of this methodology in comparison to more traditional ones, it is asserted (Romero-Ariza, 2017) that:
■ “[…] the number of activities focused on promoting the capacity for thinking, reasoning and developing evidence-based arguments is a positive indicator of the understanding of scientific ideas on the part of students exposed to inquiry-based learning (p. 291). ■ […] unguided inquiry does not facilitate learning, whereas important benefits are obtained when the process is duly assisted bythe teacher (p. 292). ■ […] the size of the moderated effect on the learning of concepts and a greater impact on the development of inquiry skills […] (p. 292). ■ […] the size of the effect of the inquiry depends on the type of inquiry activities that have taken place in the classroom, on the level of guidance or support offered by the teacher and the type of learning result measured (p. 292). ”
To shed light on this matter, this paper contains a systematic review of the articles published over the last decade in Spanish experimental science magazines, following the methodology described below:
Method
The systematic review presented in this work has been carried out in accordance with:
Revista de Educación, 381. July-September 2018, pp. 249-274 257 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
(1) The definition provided by Higgins & Green (2008), which is conceived as the review of one or more clearly formulated questions, using systematic and explicit methods for identifying and selecting the investigations referring to it, and analysing the data of those studies included in the review. (2) The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement, whose purpose is the evaluation of the quality of systematic reviews and meta-analyses (Sotos-Prieto, Prieto, Manera, Baladia, Martínez-Rodríguez & Basulto, 2014).
The search began at the beginning of October 2017, and was carried out on the main Spanish magazines specific to the field of Experimental Science Teaching. Enseñanza de las Ciencias (hereafter, EC), Revista Eureka sobre Enseñanza y Divulgación de las Ciencias (REEDC), Revista Electrónica de Enseñanza de las Ciencias (REEC), Didáctica de las Ciencias Experimentales y Sociales (DCES) and Alambique. We reviewed the numbers published between 2007 and 2017 (both inclusive), except for the special EC editions published in 2009 and 2013, as they were communications presented in the VIII and IX International Congresses on Research into EST, respectively.
Article selection procedure
The articles were selected independently by five of the authors of this study, and this was carried out via the reading of the title, abstract and keywords of each work. We included qualitative and quantitative studies, as well as experiences and teaching proposals as long as one of their topics dealt with inquiry-based teaching topics (or, failing this, research- based teaching). In this regard, the inclusion criteria we employed were:
(1) Articles published in Spanish magazines specialised in the field of Experimental Science Teaching. (2) Articles published between 2007 and 2017, both inclusive. (3) Works that alluded, either in their title, abstract or keywords, to inquiry-based teaching. (4) Studies that showed educational implications that made reference to IBSE.
258 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
The selection process for the works was carried out in two stages. Firstly, each one of the five authors charged with the task independently reviewed the magazine allocated to them, detailing the number of articles published per year and that of those selected from each magazine. Then, two of the authors dedicated to this task reviewed the five magazines for a second time, this time as a group, obtaining a consensus of 91%. The disagreements obtained in the second stage were consensually resolved by both researchers.
Data extraction procedure
In order to analyse and extract the data from the articles, we followed the same procedure outlined in the previous section. Thus, the selected works were distributed amongst the same five authors charged with the selection, for which they carried out the process independently. Once the data extraction was finalised based on the following items: (a) author/s and year, (b) educational stage, (c) type of study, (d) methodological design and (e) educational implications, they were reviewed jointly by the same two authors responsible for the final selection, obtaining a consensus of 84% with regards to the initial data extraction. The disagreements were resolved consensually by both researchers.
Results and Discussion
Following the PRISMA statement recommendations, we have created the flowchart in Figure II, which shows the selection and inclusion process followed in this study, according to the actions of identification, sifting, appropriateness and inclusion. There was a review of 1941 articles published between 2007 and 2017 in the Spanish EST-specific magazines. Following the reading of the title, abstract and keywords of each study and the application of the three first inclusion criteria, 62 articles were selected, with a total of 1879 having been excluded that did not deal with (at least explicitly in the parts analysed) IBSE. The end of the selection process was carried out through the complete reading of the 62 preselected articles, of which 7 were excluded for not fulfilling the fourth inclusion criterion. Finally, a
Revista de Educación, 381. July-September 2018, pp. 249-274 259 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
sample of 55 works remained, representing scarcely 3% of the reviewed works. With this complete, we proceeded to provide a response to the research questions.
FIGURE II. Flowchart in accordance with the PRISMA statement.
Evolution of the production of articles in Spain on IBSE
How much research into IBSE is carried out? If we compare the number of articles obtained with that of other similar works, such as the 130 studies
260 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production included in the review of Minner et al. (2010) or the 164 selected by Alfieri et al. (2011), we can state that the production of works published in national magazines that refer to IBSE is lower than the international figures, given that this encompasses a much more extensive geographical scope, as it includes articles written in English and, normally, from foreign countries and magazines. The distribution of this production is presented in Graphic I.
GRAPHIC I. Evolution of the production of articles in Spain on IBSE.
According to what Graphic I shows, the fluctuation of articles referring to IBSE published in Spain is evident. However, a certain increase in production is perceived, which is endorsed by the fact that 75% (N=41) of the selected articles have been published in the last five years. This could respond to the recommendations on the employment of IBSE issued both at an international (Ofsted, 2011; NRC, 2012) and national level (Couso, Jiménez, López-Ruiz, Mans, Rodríguez, Rodríguez & Sanmartí, 2011). Of the total number of works selected, 84% were developed in Spain, with the remaining 16% originating from Central and South American countries. Thus, the authors of the nine works published in Spain but written outside the country belong to educational institutions in Brasil (5), Columbia (1), Argentina (1) and Bolivia (1). In this sense, it is important to note that no work has been found in which educational institutions from different countries collaborate.
Revista de Educación, 381. July-September 2018, pp. 249-274 261 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
Who, on what educational stage and how is research on IBSE carried out?
Who researches IBSE? We have identified 82 authors among the 55 articles selected, which represents a collaboration index of 1.49. Contributing to this fact is the number of studies by a single author, which occurs in 33% (N=18) of the included articles, being lower than the average obtained for Social Sciences in Spain (2 authors per work), and that established in Science Teaching (1.81) (Anta & Pérez, 2007). Taking a closer look at the analysis of who researches IBSE, we present Tables II and III.
TABLE II. List of authors and number of works published on IBSE
Classification No. authors Percentage Authors with 1 work 58 71% Authors with 2 works 15 18% Authors with 3 works 4 5% Authors with 4 works 4 5% Authors with 5 or more works 1 1% Total 82 100%
As can be observed in Table II, 71% of authors have only one work attributable to them, whereas a mere 11% boast three or more. In this regard, we could state that this line of research lacks big producers (always basing ourselves on the publications in Spanish EST-specific magazines), given that the most productive, J. Domènech Casal, has published a total of 8 articles in the time period analysed. Furthermore, the number of authors to have participated in the creation of 3 or more works stands at 9 (Table III). This situation, together with the scarce number of articles on IBSE produced in Spain, could point to Spanish educational research into IBSE being at an initial stage, as only 3% of the articles published in the magazines analysed between 2007 and 2017 focus on this line of investigation.
262 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
TABLE III. Authors with the highest production in Spain on IBSE
Author No. Articles J. Domènech Casal 8 M. Martínez Chico R. López-Gay Lucio-Villegas 4 M. R. Jiménez Liso B. Crujeiras Pérez A. García Carmona C. Ferrés Gurt 3 M. Romero Ariza I. M. Greca
On what educational stage is IBSE research carried out? We find the responses to this question in Table IV. Regarding the educational stage researched, works have been identified on: Primary Education, Secondary Education, University and Graduates (referring to ongoing professional training). However, the attention given to each educational stage is far from equitable, with Secondary Education covering almost 50% of the studies included in the review, and the rest distributed amongst Primary Education (12%), University (29%) and Graduates (4%). We also find works that fail to specify the educational stage on which the study was based or, where appropriate, to which it is directed in the case of the teaching proposals (2%), and others in which it does not proceed to speak about the educational stage as they are literature review studies (7%). Accordingly, we can make reference to the conclusions drawn by Pro & Rodríguez (2010), given that EST focuses once again on the Secondary Education stage, with a marginal or negligible number of studies on the Early Childhood and Primary Education stages, which could be due to the professional careers of the authors of the selected studies, as many of them have been or are Secondary Education teachers. This goes against the recommendations that come to use from other lines of research, which attribute many benefits to beginning scientific education in initial school years (Eshach & Fried, 2005). The results obtained show that the presence of IBSE methodology is almost non-existent in Spanish science classrooms (Cortés et al., 2012). This is undoubtedly due in principle to the fact that teachers are unsure of how to implement the process and methods habitually used by scientists in the classroom and maintain, on many occasions, inappropriate
Revista de Educación, 381. July-September 2018, pp. 249-274 263 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
concepts regarding what IBSE is, such as identifying inquiry as straying from curricular planning (Keys & Kennedy, 1999).
TABLE IV. What educational stages is IBSE research carried out on?
Item Variables N Percentage Primary Education 7 12% Secondary Education 26 46% University 16 29% Educational stage Graduates (ongoing training) 2 4% Not specified 1 2% Not relevant 4 7% Empirical study 23 42% Type of study Theoretical study 32 58% Experimental 6 11% Ex post-facto 3 5% Case study 12 21% Ethnographic 1 2% Design Research-action 2 4% Literary review 4 7% Teaching experience 23 41% Teaching proposal 5 9%
Regarding how IBSE research is carried out, we have focused on two dimensions: study type and design. For study type, it is observed that almost 60% (N=32) are theoretical works which describe teaching experiences or proposals, or present a theoretical contribution through narrative literary reviews. In consequence, the number of empirical studies selected scarcely exceeds twenty. As regards design, the most repeated corresponds to that concerning the description of a teaching experience (N = 23), whereas the empirical design most used is the case study (N= 12). In addition, the number of studies with experimental design is extremely reduced; their presence barely exceeds 10% of the articles selected.
264 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
In the light of these studies, attention should be drawn to the scarcity of empirical studies referring to IBSE methodology and published in Spanish EST magazines, given that it has a significant impact on the nature of the results and educational implications obtained, on the whole resulting from sensations or opinions lacking the rigour and systematicity necessary for their possible replication and generalisation. In this regard, it is worth pointing out that it would be necessary to implement IBSE methodology in classrooms, but affording the methodological rigour, systematicity and reflection, in order to increase the number of empirical studies in this line of research. This could be done either via an experimental evidence-based design, which allows the quantifying of effects (Bisquerra, 2014), or via qualitative designs such as the case study or action research, which are no less rigorous and have the positive aspect of improving the quality of teaching practice (in science teaching, in this case) from this practice itself (Elliot, 1993).
What educational implications are demonstrated regarding IBSE?
The educational implications reflected by each of the selected articles are summarised in Table V, in which each work appears referenced via the number assigned to it in the Annex. We have thus used six criteria to classify them, referring to: (1) skills, (2) emotions, motivation and attitudes, (3) knowledge acquisition and transfer, (4) nature and image of science, (5) teachers and (6) student difficulties.
Revista de Educación, 381. July-September 2018, pp. 249-274 265 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
TABLE V. Summary of educational implications linked to IBSE.
Referring to N* Educational implications Promotes student autonomy (6, 7, 23). 3 Encourages cooperative work amongst students (1, 6, 18, 22, 23, 24, 43). 7 Competencies Develops language skills (16, 23, 43). 3 Favours the development of skills relating to scientific methodology (12, 13, 16, 8 18, 22, 33, 45, 48). 4 Creates positive attitudes towards science (6, 24, 33, 44). 15 Increases motivation and satisfaction towards science learning (6, 11, 14, 16, 23, 30, Emotions, 34, 35, 36, 37, 43, 46, 48, 52, 55). motivation and 2 Students value the employment of this methodology positively (32, 45). attitudes 1 Increases student self-esteem regarding science learning (24). 1 Does not affect student motivation and emotional state. Improves the acquisition of content (7, 12, 14, 15, 16, 23, 30, 38, 41, 42, 46). 11 Helps to achieve significant learning (10, 14, 15, 50, 52). 5 Encourages reflection and critical thinking during the teaching-learning process (6, 9 10, 11, 17, 21, 22, 24, 46, 55). Fosters active participation of students (24, 33, 34, 49, 55). 5 Increases dialogue between teacher and students (5). 1 Knowledge Takes students’ interests and previous ideas into account (11). 1 acquisition and Favours the identification and modification of mistaken ideas for those adapted to 4 transfer scientific knowledge (15, 24, 47, 50). Helps to manifest the different current social-scientific problems and adapts to 2 the needs of students, where uncertainty generated by highly accelerated change dominates (25, 26). 6 Allows students to become aware of the usefulness of scientific knowledge in their daily lives (14, 28, 29, 43, 44, 46). 1 Facilitates the transfer of ideas and knowledge from one context to another (15). Contributes to a closer presentation of the production, development and impact of 4 Nature and image scientific knowledge (53, 4, 5, 35). of science Allows the modification of negative ideas about the image of science and affords 3 value to scientific work (14, 29, 31). Training future IBSE teachers could affect future educational innovation, improving their knowledge on this teaching strategy and generating more confidence and 9 better teacher performance (9, 27, 31, 38, 39, 40, 41, 48, 49). Requires more dedication on the part of the teacher than traditional teaching (33, 36). 2 The role of the teaching guide during the inquiry process is essential (19, 48). 2 University teachers consider the employment of IBSE in initial teacher training as 1 a priority (39). Teachers 3 Teachers in initial training positively value this teaching strategy (1, 3, 51). 2 Teachers in initial training find it difficult to design teaching sequences based on 1 inquiry (3, 51). Trainee teachers have little idea of how to create, validate and evolve scientific knowledge, do not know basic techniques or procedures via which scientific 1 knowledge is developed, with all of this affecting the image they may transmit (54). NPTAI is a useful instrument for teachers, as it is a training evaluation tool that permits the identification of student difficulties during the inquiry process (20). Lacks a clear model of scientific activity to be able to correctly carry out the 3 dynamic of inquiry (6, 8, 28). Has difficulties formulating researchable questions (21). 1 Show difficulties extracting and interpreting results, and giving a hypothesis (2). Student difficulties 1 Both oral and written expression and the identification of problematic situations 2 are, generally, student weaknesses (6, 36). Tackling inquiry activities is complex when they have little experience related to 1 this type of teaching approach (19).
* Number of articles that obtain teaching implication.
266 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
The contribution made by the IBSE methodology to the development of skills (autonomy and initiative, teamwork, language skills and those referring to research) observed in many of the works selected in this review is ratified by Bevins & Price (2016), who state that one of the strengths of this teaching focus is the promotion and development of skills and capacities linked to science. Moreover, these same authors assert the following as benefits of IBSE: improvement in self-esteem; comprehension of content and motivation; effect of the application of the new learning in the ever more complex contexts that surround them. In terms of attitude and emotions towards science, positive and negative results have been found, although on the whole the contributions collected show the effectiveness of IBSE in improving these variables (Areepattamannil 2012; McConney et al., 2014). In this same regard, the educational implications compiled point to an improvement in the acquisition of content on the part of students, and the favouring of significant learning, as a result of reflection, the evaluation of alternative ideas, reasoning (Pedaste et al., 2015) and the active participation of students (Bevins & Price, 2016); all of these actions are habitually promoted by the educational proposals and experiences analysed. The effects generated by IBSE methodology described above, all of them positive, foster a classroom atmosphere that is relaxed, pleasant and, as a result, ideal for science learning (Abril, Romero-Ariza, Quesada & García, 2014). In turn, this agreeable benefit is key to the promotion of positive attitudes towards school and learning (Krüger, Formichella & Lekuona, 2015). Thus, given the deterioration of attitudes towards science experienced by the students of today (Vázquez & Manassero, 2008), we could consider this teaching focus as an appropriate means for its reversal. There are, however, a number of aspects to consider in the putting into practice of this teaching focus, as we can see in those educational implications that refer to teachers (Table V). The role, therefore, taken on by the teacher during this teaching-learning process, his or her prior experience and knowledge on IBSE and its implication are determining factors in the effectiveness of this teaching strategy, converting the teacher into a capable professional (Reoyo, Carbonero & Martín, 2017). Such is the case that some authors have questioned the IBSE methodology, alleging that:
Revista de Educación, 381. July-September 2018, pp. 249-274 267 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
(1) Students cannot learn by themselves through inquiry (Kirschner, Sweller & Clark, 2006), as there is evidence to suggest that better learning results are obtained through guided inquiry (Minner et al., 2010; Furtak et al., 2012; Lazonder et al., 2016). (2) Teachers lack preparation (Forbes & David, 2010), given the fragmented, shallow and unsound scientific knowledge available to them (Murphy et al., 2007). (3) The difficulties faced by teacher trainers in designing and executing inquiry-based sequences (Lucero et al., 2013; Crujeiras & Puig, 2014), a line along which there are already a number of training proposals to attempt to overcome such deficiencies (Martínez- Chico, Jiménez-Liso & López-Gay, 2015; Vílchez & Bravo, 2015; Greca, Meneses & Diez, 2017).
Finally, weighing up the educational implications that are more and less encouraging, we can state that inquiry-based teaching is an effective way of teaching and learning science, highlighting the increase in motivation/satisfaction in terms of learning science and the acquisition of content on the part of students (N = 15 y N = 11 respectively) as the most relevant implications. Lederman, Lederman & Antink (2013) coincide on this point, as long as the activity design is appropriate, which is necessarily arrived at through relevant teacher training.
Conclusions
This review paper provides, on the one hand, a perspective of the development of research on IBSE methodology in the field of EST, focusing on publications in Spanish magazines; on the other hand, it compiles the main educational implications referring to this methodology. We can thus conclude the following: First. Scientific production on IBSE methodology in EST-specific Spanish magazines is scarce, despite the existence of authors from other countries who have disseminated a number of studies in Spain. This adds to the complexity of highlighting a researcher or research group with noteworthy production along this line of investigation. Attention should also be drawn to the meagre degree of collaboration in the production of the articles. Notwithstanding this, there has been a favourable evolution
268 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production in terms of the volume of articles that tackle this subject in the last five years. We could therefore argue that research in Spain onIBSE methodology is incipient. Second. Attention paid to the different educational stages is very unequal, with the highest number of studies focusing on secondary education and university, which could be conditioned by the professional trajectory of the authors, who work or have worked as secondary school teachers. In consequence, EST research pays little attention to the primary education stage and, in this case, we have found no study with a focus on early childhood education. Third. There are more theoretical than empirical studies. The most frequent design used for these theoretical works has been teaching experience, with the case-study design generally being used for empirical studies. As a result, a channel for promoting the extraction of more rigorous and generalizable conclusions would be an increase in the number of experimental studies (from the quantitative point of view) or a greater use of action research (from the qualitative perspective). Fourth and last. The educational implications referring to IBSE methodology are encouraging in terms of the effects generated in students and teachers in training, as both enable a solid scientific literacy based on positive attitudes and emotions, and on a rejuvenated image of science. This teaching focus, however, faces a number of important challenges.
■ Regarding teachers: – Improving knowledge of future teachers on the nature of science. – Reducing their difficulties in carrying out IBSE methodology, difficulties which are related to the insecurity created bythe aforementioned points and the lack of experience putting this methodology into practice. ■ Regarding students: – Promoting science learning based on doing science, in the interest of correcting deteriorated attitudes, negative feelings and emotions and false assumptions regarding science.
The results obtained seem to suggest that IBSE could be a good teaching-learning strategy for science, and that there is also a long path to tread. Studies such as those presented in this article may show the
Revista de Educación, 381. July-September 2018, pp. 249-274 269 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
way. Given its limitations, as it focuses on research in a national sphere (Spain), the line to follow would be to firstly widen the study to the international sphere, which is the intention in future reviews.
Acknowledgements
This study falls within the project “Design, evaluation and dissemination of workshops based on inquiry for Primary Education (ref: PPJI_B-01)” funded by the University of Granada. We also wish to express our thanks to the Autonomous Government of Andalusia and the European Social Fund for providing the funding for contract no. 6161, destined towards the contracting of young research personnel, to the Ministry of Education, Culture and Sport for the award of grant FPU15/04972, and to the HUM- 613 research group for participating in their training.
Annex: www.didacticacienciasugr.es/docs/misc/Anexo-Rev-Sis-Nacional.pdf
Bibliographical references
Abril, A. M., Romero-Ariza, M., Quesada, A., & García, F. J. (2014). Creencias del profesorado en ejercicio y en formación sobre el aprendizaje por investigación. Revista Eureka sobre enseñanza y divulgación de las Ciencias, 11(1), 22-33. Alfieri, L., Brooks, P.J., Aldrich N.J., & Tenenbaum H.R. (2011). Does discovery-based instruction enhance learning? Journal of Educational Psychology, 103, 1–18. Anderson, R.D. (1996). Study of curriculum reform. Washintong DC: U.S. Goverment Printing Office. Anta, C. & Pérez, J. M. (2007, November, 21-23). La producción científica sobre Didáctica de las Ciencias. IV Congreso Comunicación Social de la Ciencia. Madrid: CSIC. Areepattamannil, S. (2012). Effects of inquiry-based science instruction on science achievement and interest in science: Evidence from Qatar. The Journal of Educational Research, 105(2), 134–146.
270 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
Barrow, L. H. (2006). A brief history of inguiry: from Dewey to Standards. Journal of Science Teacher Education, 17, 265-278. Bevins, S. & , G. (2016). Reconceptualising inquiry in science education. International Journal of Science Education, 38(1), 17-29. Bisquerra, R. (Coord.) (2014). Metodología de investigación educativa. Madrid: La Muralla. Cañal de León, P. (1987). Un enfoque curricular basado en la investigación. Investigación en la Escuela, 1, 43-50. Cobern, W.W., Schuster, D., Adams, B., Applegate, B., Skjold, B., Undrieu, A., Loving, C.C., & Gobert, J.D. (2010). Experimental comparison of inquiry and direct instruction in science. Research in Science & Technological Education, 28(1), 81-96. Cortés, A.L., de la Gándara, M., Calvo, J.M., Martínez, M.B., Gil, M.J., Ibarra, J. & Arlegui, J. (2012). Expectativas, necesidades y oportunidades de los maestros en formación ante la enseñanza de las ciencias en la educación primaria. Enseñanza de las Ciencias, 30(3), 155-176. Couso D., Jiménez M. P., López-Ruiz J., Mans C., Rodríguez C., Rodríguez J.M., & Sanmartí, N. (2011). Informe ENCIENDE: Enseñanza de las Ciencias en la Didáctica escolar para edades tempranas en España. Madrid: Rubes Editorial. Couso, D. (2014). De la moda de “aprender indagando” a la indagación para modelizar: una reflexión crítica. Ponencia a los XXVI Encuentros de Didáctica de las Ciencias Experimentales, Huelva. Crawford, B.E. (2007). Learning to teach science as inquiry in the rough and tumble of practice. Journal of Research in Science Teaching, 44, 613–642. Crujeiras, B. & Puig, B. (2014). Trabajar la naturaleza de la ciencia en la formación inicial del profesorado planificando una investigación. Educació Química, 17, 55-61. Cuevas, P., Lee, O., Hart, J. & Deakort, R. (2005). Improving science inquiry with elementary students of diverse backgrounds. Journal of Research in Science Teaching, 42, 337-357. Elliot, J. (1993). El cambio educativo desde la investigación-acción. Madrid: Morata. Eshach, H. & Fried M. N. (2005). Should science be taught in early childhood? Journal of Science Education and Technology, 14(3), 315- 336.
Revista de Educación, 381. July-September 2018, pp. 249-274 271 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
Forbes, C. T. & Davis, E. A. (2010). Curriculum design for inquiry. Pre- service elementary teachers’ mobilization and adaptation of science curriculum materials. Journal of Research in Science Teaching, 47, 820-839. Furtak, E.M., Seidel, T., Iverson, H., & Briggs, D.C. (2012). Experimental and Quasi-Experimental Studies of Inquiry-Based Science Teaching. Review of Educational Research, 82(3), 300-329. Gil-Pérez D. (1990). Un modelo de resolución de problemas como investigación. Madrid: Ministerio de Educación y Ciencia – Lábor. Gil-Pérez D. (1993). Contribución de la historia y la filosofía de las ciencias al desarrollo de un modelo de enseñanza/aprendizaje como investigación. Enseñanza de las Ciencias, 11(2), 197-212. Greca, I. M., Meneses, J. A., & Diez, M. (2017). La formación en ciencias de los estudiantes del grado en maestro de Educación Primaria. Revista Electrónica de Enseñanza de Las Ciencias, 16(2), 231–256. Hazelkorn, E. (2015). Science Education for Responsible Citizenship. Comisión Europea. http___ec.europa.eu_research_swafs_pdf_pub_ science_education_KI-NA-26-893-EN-N Higgins, J. P. T., & Green, S. (2008). Cochrane handbook for systematic reviews of interventions. West Sussex, England: Cochrane Collaboration & Wiley. Keys, C.W. & Kennedy, V. (1999). Understanding inquiry science teaching in context: A case study of an elementary teacher. Journal of Science Teacher Education, 10(4), 315-333. Krüger, N., Formichella, M. M., & Lekuona, A. (2015). Más allá de los logros cognitivos: la actitud hacia la escuela y sus determinantes en España según PISA 2009. Revista de Educación, 367, 10-35. Kirschner P.A., Sweller J., & Clark R.E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry- based teaching. Educational Psychologist, 41, 75–86. Lazonder, A.W., & Harmsen, R. (2016). Meta-Analysis of Inquiry-Based Learning Effects of Guidance. Review of Educational Research, 20(10), 1-38. Linn, M. C., Davis, E.A. & Bell, P. (2004). Internet environments for science education. New Jersey: Lawrence Erlbraum Associates. Lucero, M., Valcke, M. & Schellens, T. (2013). Teachers’ beliefs and self- reported use of inquirí in Science Education in Public Primary Schools. International Journal of Science Education, 35 (8), 1407-1423.
272 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
Martínez-Aznar, M. (2017). La enseñanza-aprendizaje de las ciencias: «entre indagación y problemas». Comunicación personal. Martínez-Chico, M. (2013). Formación inicial de maestros para la enseñanza de las ciencias. Diseño, implementación y evaluación de una propuesta de enseñanza. Tesis doctoral. Universidad de Almería. Martínez-Chico, M., Jiménez Liso, M. R., & López-Gay, R. (2015). Efecto de un programa formativo para enseñar ciencias por indagación basada en modelos , en las concepciones didácticas de los futuros maestros. Revista Eureka Sobre Enseñanza Y Divulgación de Las Ciencias, 12(1), 149–166. McConney A., Oliver M.C., Woods-McConney A., Schibeci R. & Maor D. (2014) Inquiry, Engagement, and Literacy in Science: A Retrospective, Cross-National Analysis Using PISA 2006. Science Education, 98(6), 963-980. McDonald, S. & Butler Songer, N. (2008). Enacting classroom inquiry: Theorizing teachers’ conceptions on science teaching. Science Education, 27, 45-60. Minner, D.D., Levy, A.J. & Century, J. (2010). Inquiry-Based Science Instruction—What Is It and Does It Matter? Results from a Research Synthesis Years 1984 to 2002. Journal of Research in Science Teaching, 47(4), 474–496. Murphy, C., Neil, P. & Beggs, J. (2007) Primary science teacher confidence revisited: ten years on. Educational Research, 49(4), 415-430. National Research Council. (2012). A Framework for K-12 science education: Practices crosscutting concepts, and core ideas. Washington, DC: National Academy Press. Ofsted (2011). Successful science: An evaluation of science education in England 2007 – 2010. Manchester, UK: Ofsted. Pedaste M., Mäeots M., Siiman L.A., De Jong T., Van Riesen S.A., Kamp E.T., & Tsourlidaki E. (2015). Phases of inquiry-based learning: Definitions and the inquiry cycle. Educational research review, 14, 47-61. Perales, F.J., & Ayerbe, J. (2016). El trabajo por proyectos y por resolución de problemas en Educación Ambiental: análisis y tendencias. Comunicación a los XXVII Encuentros de Didáctica de las Ciencias Experimentales. Badajoz (7-9 de Septiembre, 2016). Pro, A., & Rodríguez, (2010). Aprender competencias en una propuesta para la enseñanza de los circuitos eléctricos en Educación Primaria. Enseñanza de las Ciencias, 28(3), 385-406.
Revista de Educación, 381. July-September 2018, pp. 249-274 273 Received: 05-12-2017 Accepted: 16-03-2018 Aguilera, D., Martín-Páez, T., Valdivia-Rodríguez, V., Ruiz-Delgado, A., Williams-Pinto, L., Vílchez-González, J. M., Perales-Palacios, F. J. Inquiry-based Science Education. A systematic review of Spanish production
Reoyo, N., Carbonero, M. A., & Martín, L. J. (2017). Características de eficacia docente desde las perspectivas del profesorado y futuro profesorado de secundaria. Revista de Educación, 376, 62-86. Rocard, M. (Chair), Csermely, P., Jorde, D. Lenzen, D., Walberg-Henriksson, H. & Hemmo, V. (2007). Science Education now: A renewed Pedagogy for the future of Europe. Comisión Europea. https://sede.educacion. gob.es/publiventa/PdfServlet?pdf=VP15136.pdf&area=E Sotos-Prieto, M., Prieto, J., Manera, M., Baladia, E., Martínez-Rodríguez, R. & Basulto, J. (2014). Ítems de referencia para publicar Revisiones Sistemáticas y Metaanálisis: La Declaración PRISMA. Revista Española de Nutrición Humana y Dietética, 18(3), 172-181. Vázquez, A. & Manassero, M.A. (2008). El declive de las actitudes hacia la ciencia de los estudiantes: un indicador inquietante para la educación científica. Revista Eureka Enseñanza y Divulgación de las Ciencias, 5(3), 274-292. Vílchez, J. M., & Bravo, B. (2015). Percepción del profesorado de ciencias de educación primaria en formación acerca de las etapas y acciones necesarias para realizar una indagación escolar. Enseñanza de Las Ciencias, 33(1), 185–202. Windschitl, M. (2003). Inquiry projects in science teacher education: What can investigative experiences reveal about teacher thinking and eventual classroom practice? Science Education, 87(1), pp. 112-143. Yeomans, E. (2011). Inquiry-based learning – what is its role in an inspiring science education? En Perspectives on Education: Inquiry- based learning (p. 3). Wellcome Trust: Londres.
Contact address: David Aguilera Morales. Facultad de Ciencias de la Educación, Campus Universitario de Cartuja, C.P. 18071 (Granada), Granada. E-mail: [email protected]
274 Revista de Educación, 381. July-September 2018, pp. 249-274 Received: 05-12-2017 Accepted: 16-03-2018 Do experimentation outreach programs improve attitudes towards school science?1
¿Mejoran los proyectos de divulgación con experimentación la actitud hacia las clases de ciencias?
DOI: 10.4438/1988-592X-RE-2017-381-389
Raquel Fernández-Cézar Natalia Pinto-Solano Universidad de Castilla la Mancha Marta Muñoz-Hernández Universidad Rey Juan Carlos
Abstract There is an increasing interest in the affective domain in science education, and particularly in the attitudes towards school science, as a possibility to counteract the present decline of scientific vocations observed in the last few years. Thus, this study aims to evaluate the effect of an experimental outreach program on the attitude towards school science. The possible influence of factors such as educational stage, sex and geographical context (rural or urban) is taken into account. This is a quantitative, cross-sectional study with a brief questionnaire being administered before and after the intervention to measure student attitude. The sample consists of 380 students from the pre-university schools participating in the project. Statistical analysis was carried out using the SPSSv22 computer software. The results obtained after comparing the pretest and posttest show a more positive attitude towards school science in the posttest (t=5.46; df=299; p<.01). This increase is more prevalent among elementary school students (t=-6.61; df=210; p<.00), and among students from rural areas (p<.00). No significant difference based on sex is found (p=.38 pretest; p=.06 posttest).
(1) The authors acknowledge the financial support of the Spanish Foundation of Science and Technology, FECYT, in 2012 and 2014 under the contracts FCT-12-4221 y FCT-14-8844.
Revista de Educación, 381. July-September 2018, pp. 275-295 275 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
The conclusion of our evaluation is that this outreach program has a more positive influence on elementary students than on those in secondary education. Therefore, interventions regarding the future selection of science related subjects should take place in the early stages of education, such as the elementary level, where a more positive increase is obtained, and to keep the actions in a more intense and long-term format. With regard to the increase in positive attitudes among students living in rural environments found in this study, further research is needed to confirm whether or not these results can be generalized.
Key words: Attitude towards school science, Elementary Education, Secondary Education, Outreach program
Resumen Existe un interés creciente por el dominio afectivo en la didáctica de las ciencias, y en concreto por la actitud hacia las clases de ciencias como posible acción contra el declive de vocaciones científicas que se viene observando en los últimos años. Por ello, este trabajo valora el efecto de un programa divulgativo basado en la experimentación sobre la actitud hacia las clases de ciencias. En dicha valoración se tienen en cuenta factores como la etapa educativa, el sexo y el entorno del centro (rural o urbano). La metodología utilizada es cuantitativa, el estudio es transversal, y el instrumento para medir la actitud es un cuestionario breve que se administra como pretest y postest. La muestra está compuesta por 380 alumnos de los centros escolares no universitarios participantes en el proyecto. El análisis estadístico se realiza con el paquete informático SPSSv22. Los resultados obtenidos al comparar el pretest y el postest muestran: una actitud más positiva hacia las ciencias en la escuela en el postest (t=5.46; gl=299; p<.01), mayor en el alumnado de la etapa de educación primaria (t=-6.61; gl=210; p<.00), y en el alumnado de zonas rurales (p<.00); no se aprecia influencia del sexo en el efecto del programa sobre la actitud (p=.38 antes; p=.06 después). Se concluye que la actuación desarrollada tiene una influencia más positiva en los estudiantes de la etapa de educación primaria que en los de educación secundaria. Se propone que las acciones que pretendan actuar sobre los futuros itinerarios científicos a seguir por los estudiantes deberían iniciarse en la etapa de primaria, donde se obtiene un efecto más positivo, y prolongarse e intensificarse en educación secundaria. Es necesaria una investigación más profunda para confirmar (o no) si en las zonas rurales el alumnado presenta una actitud hacia las ciencias más positiva de forma generalizada.
Palabras clave: Actitud hacia las clases de ciencias, Educación Primaria, Educación Secundaria, Programa de divulgación
276 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
Introduction
The study of attitudes started in the field of social psychology (Allport, 1935, cited in di Martino & Zan, 2010) with the purpose of predicting individual choices related to, among other things, voter intention and consumer purchases. In addition to being a modifiable trait, attitude entails behavioral, cognitive and affective components (Heckman, Stixrud & Urzua, 2006) that can be positive or negative in the sense that it can express attraction or aversion (Eiser, 1989). Within education, attitude is a construct that, according to the existing paradigm, has been linked to three different conceptual models: unidimensional, two dimensional (affective and cognitive) and three dimensional (behaviors, emotions and cognitions). The evidence supported by structural equation studies have confirmed that the model of two components is the one which fits best and has the best meaning; therefore, concluding that attitude consists of affective and cognitive variables, created from subjects’ personal experience (Vázquez, 2013). The present work uses this bi-dimensional model considering that the attitude towards school science encompasses cognitions and affects. In the field of science education there has not existed a tradition for the development of attitude models, but rather it has been more common to measure them by different instruments. It is not until this century that efforts have been devoted to provide this field with the required theoretical foundations (Alsop, 2005; Saleh & Khine, 2011). Also in this century, some reviews of the existing literature have been carried out regarding attitudes towards science, where diverse instruments have been used (Osborne, Simon, & Collins, 2003; Mellado et al., 2013; Potvin &Hasni, 2014). In the aforementioned reviews, attitude towards school science is recognized within the taxonomy of attitude towards science. Therefore, the authors of these reports proport that a part of the student’s attitude towards science is influenced by the science learnt at school, or school science. Since Vázquez- Alonso and Manassero- Mas (2011), it is necessary to incorporate attitude towards school science in whatever theoretical model intended to address and/or improve the teaching and learning of school science. In this imbrication line of attitude and the teaching-learning of science, the results about the experimentation as part of the instruction are contradictory. While Vázquez- Bernal and Jiménez- Perez (2016) discuss the association between the use of experimentation
Revista de Educación, 381. July-September 2018, pp. 275-295 277 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
in the classroom and the pupils’ attitude towards school science, the work of Fernández Cézar and Solano Pinto (2017) report a lack of that. The majority of studies on the attitude towards science use questionnaires as instruments (Rubba, & Harkness, 1993; Manassero-Mas, Vázquez-Alonso, & Acevedo-Díaz, 2001; Manassero, Vázquez, & Acevedo, 2003; Siegel, & Ranney, 2003; Chen, 2006; Vázquez Alonso, Acevedo Díaz, Manassero Mas, & Acevedo Romero, 2006; Vázquez Marbà-Tallada, & Márquez Bargalló, 2010; de Pro Bueno & Pérez Manzano, 2014; Kennedy, Quinn & Taylor, 2016). The time-information relationship is optimized with these instruments, as Vázquez, Manassero and Acevedo (2006), and more recently Kennedy, Quinn and Taylor (2016) report. Among the questionnaires to measure attitude towards sciences, there exits some with the focus on science, technology and society relation, STS, like the COCTS (Bennássar, Váquez, Manassero & García-Carmona, 2010), composed by questions taken from another questionnaire, Views on Science, Technology and Society, VOST, (Aikenhead, & Ryan, 1992). In addition to this, there are broad range studies, such as The Relevance of Science Education, ROSE (Schreiner & Sjøberg, 2004), carried out with students from all around Europe, who are between 15 and 17 years of age. The results on Spanish students are included in several papers (Alonso, & Mas, 2007; Alonso, & Mas, 2009; ROSE, A. C. D. P., 2005). But all these studies use very long questionnaires, with complex formulated sentences, which result in comprehension difficulties even for the target students. In addition to that, these questionnaires contain an enormous number of items, 657 the VOST, which impedes their autonomous use by young students, who are the target group of this study. Having so many items can cause confusion and lead students to neglecting items, resulting in a big number of wrong answers due to the students’ mental fatigue, and, in the end, is very time consuming for the researcher. (Abad, Olea, Ponsoda & García, 2011). Therefore, the most appropriate questionnaire will be the one which has a reasonable number of items/time ratio. Aligned with this objective, some current studies have been found addressing the analysis of attitudes towards school science in pupils younger than 12 (Kennedy, Quinn & Taylor, 2016; Fernández Cézar & Solano Pinto, 2017) where the used instruments have a low number of items with simple sentences that students can complete in a quick and autonomous way. In these instruments there are items referring to the three fundamental aspects of attitude, as in di Martino and Zan (2010): I
278 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science? like / don’t like (interest-affective), because I can / cannot do (perception of the own competence-cognitive), because sciences are / aren’t (beliefs on sciences-cognitive). Attitude is considered to be a set of cognitions and emotions modifiable through actions, and thus attitude towards school science is susceptible to improvement. This research project focuses on an outreach project aimed at fostering scientific vocations, accessible at www.nanomadrid. es, consisting in bringing to pre-university education centers student- involved experiments carried out by Spanish university and research centers’ researchers. The experiments requested by the schools deal with contents within the curriculum associated with the students’ particular educational stage, such as chemical reactions, changes of state, properties of materials (magnetism, electricity…), superconductivity, etc. The contents and methodology are tailored in such a way as to take into account the participants’ educational and developmental stage, always keeping in mind the essence of learning through discovery, and by involving the students in the implementation of the scientific method. The approach entails the students’ involvement in the experiment, enabling them to reflect on the subject matter, as there is no learning without the students creating their own knowledge through reflection. There are previous studies regarding he influence of different interventions on pre-university students’ attitude towards school science (Barmby, Kind, & Jones, 2008; Cardona et al., 2012). More current works are dealing with the measurement of this attitude in students younger than 12 years of age, but the analyzed interventions involve instructional changes bringing about the use of Webquests (Unal, & Altiparmak, 2013), programs related to the social perception of science (Hong, Lin, Wang, Chen, & Yang, 2013), or the whole adaptation of the science school curriculum to the use of a particular technological material like Legos (Bethke Wendell, & Rogers, 2013). However, none of these studies consider the experimentation as the main intervention influencing attitude towards school science. Although one previous study showed the positive influence that experimentation as part of the instruction had on the motivation towards sciences and mathematics (Weinberg, Basile, & Albright, 2011), which would be desirable from a didactics point of view, this present study differs from the former in that the interventions are carried out by students and researchers, who are different, in the eyes of the students, from the usual classroom teacher.
Revista de Educación, 381. July-September 2018, pp. 275-295 279 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
For all the prementioneaforementioned reasons, this present paper has as a general objective of evaluating whether interventions based on experimentation with student participation in Primary Education (PE) and Secondary Education (SE) do improve the students’ attitude towards school science. Moreover, as specific objectives, this paper analyzes the possible association of this attitude to several factors: sex, educational stage (PE and SE), and the school social context (rural and urban). Taking into account the revised literature, the following results are expected: the students’ involvement in experiments improves their attitude towards school science; this improvement is greater in primary education; the highest scores correspond to male students; the biggest change in attitude towards school science will occur in students from rural areas.
Method
The methodology is quantitative: the attitude towards school science is measured through a questionnaire. The students belong to the last grade of primary education (6th PE graders, 11-12 yr), 3rd grade of compulsory secondary education (CSE, 14-15 yr), and the 1st grade of high school (HS, 16-17 yr), thus being a transversal study that provides an overall picture throughout the academic years 2014-2015 and 2015-2017.
Sample
The sample is of proximity, composed of students from the schools that participated in the outreach project Conocer la ciencia hoy abre las puertas del mañana (Fernández Cézar & Solano Pinto, 2017), which has been run by researchers from several Spanish universities and research centers in Madrid and Castilla La Mancha since 2012. Out of the 500 students who took part in the study, 380 provided valid questionnaires, and were used as the total sample. Within the sample, 154 students are girls and 146 boys; 199 live in urban areas and 101 in rural ones (villages of less than 5000 inhabitants); 227 are 6th PE graders, 68 are 3rd CSE graders, and 85 are 1st HS graders.
280 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
TABLE I. Sample description by sex and educational stages
Sex Urban Rural Total Primary 77 31 108 CSE 34 10 44 Girl High School 18 22 40 Total 139 53 92 Primary 62 57 119 CSE 18 6 24 Boy High School 19 26 45 Total 105 83 188
Instrument
The questionnaire is composed of 8 items taken from “My Science Class” domain of ROSE (Schreiner & Sjøberg, 2004), used in a previous study of primary education students (10-12 years old) (Fernández Cézar & Solano Pinto, 2017). The answers are recorded by using a Likert scale from 1 to 5, 1 meaning totally disagree (TD), 3 being neutral, and 5 meaning totally agree (TA). The items are phrased in such a way that prevents the acquiescence in the answers. In order to quantify the total attitude, the score in each item is set to 5 meaning the maximum, and 1 meaning the minimum. These are shown in table II. The internal consistency of the whole questionnaire is measured through the Cronbach alpha coefficient, which is published in a previous work with a smaller sample (Fernández Cézar & Solano Pinto, 2017). Nonetheless, it has been reevaluated using this sample achieving .810, well in the range of accepted values inside the social sciences.
Revista de Educación, 381. July-September 2018, pp. 275-295 281 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
TABLE II. Questionnaire
1. Science is a difficult subject 2. School science is interesting 3. School science has opened my eyes to interesting jobs that I could choose when I’m an adult. 4. I like school science more than other subjects 5. I think that everybody should study science at school 6. The things that I learn in school science will help me in everyday life 7. School science has shown me the importance of science in our life style 8. I would like to have as much science as possible at school
Procedure
The questionnaires were delivered to the schools after they had requested to participate in the outreach project. They were administered with the students’ informed consent before and after the intervention. The pretest was carried out by the program researchers before implementing the experiment with the students. A week later, the regular classroom teacher (tutor) who has been instructed for this purpose, administers the test again, what constitutes the posttest. The intervention was carried out during the school day, and consisted in a one to two session, during which the students worked in pairs or groups of 4 at most, implementing the experiment chosen by the school from among those offered by the program. No fixed script or guidelines were provided, but the scientific method was followed: the researchers, and sometimes the students, raised questions, and the students, with the combination of the materials provided and their own knowledge, put forth hypotheses or proposed answers, which were then evaluated by the students themselves. As every group works on the same topic, finally the reached conclusion is shared with the whole class. The data analysis was performed by using the software Statistical Package for Social Sciences, SPSS v22. All 8 items were added up to determine the total attitude, TA. It was justified by the item-total correlation coefficients, and by an exploratory factor analysis whose covariance matrix was obtained by using the Varimax and Kaiser standardization method.
282 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
In order to differentiate between total attitude towards school science before and after the participation in the project, a B and A are added to total attitude (TA) resulting in TAB and TAA respectively. These are quantitative variables, and the target factors are educational stage (primary, compulsory secondary education or high school), the sex and the region where the school is settled (rural or urban). From the normality test of quantitative variables the kind of hypothesis contrast has been depicted. The comparison of TAB and TAA was determined by comparing the means of the two sample groups. Unless otherwise mentioned, the confidence level of the statistical analysis is 95%.
Results
Total attitude before (TAB) and after the participation in the project is analyzed in this section. The values of Spearman’s ρ (rho) for the item- total correlation are shown in Table III, a.
TABLE III. a) Item-total correlation for attitude before and after the intervention
TAB TAA ITEM1 .53 .58 ITEM2 .66 .56 ITEM 3 .60 .66 ITEM 4 .60 .60 ITEM5 .52 .53 ITEM6 .64 .66 ITEM7 .57 .65 ITEM8 .56 .43 b) Factor loadings of each item to single-factor model (F1)
ITEM1 .42 ITEM2 .57 ITEM 3 .51 ITEM 4 .46 ITEM5 .64 ITEM6 .53 ITEM7 .46 ITEM8 .38
Revista de Educación, 381. July-September 2018, pp. 275-295 283 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
Exploratory factor analysis (KMO = 0.743; p <.000 for Bartlett’s test of sphericity) returns a single-factor model that explains 34.43% of the variance, with factor loadings extracted by using the maximum likelihood method for the covariance matrix, by using the Varimax-Kaiser normalization method; such values are shown in table III b. Both the item-total correlation and the exploratory factorial analysis justify the use of the total attitude, TA.
TABLE IV. Attitude towards school science before and after the intervention, mean and stan- dard deviation values. * The significance level of the hypothesis testing is included.
Mean Sd t gl p* TAB 30.15 5.56 5.46 299 <.00 TAA 32.51 5.13
On the one hand, it should be noted that, in general, there is a positive increase in attitude towards school science after the participation in the project. As the mean difference in TA is statistically significant, we can say that it is increased by 2.36 after participation in the project. On the other hand, an exploratory analysis of the quantitative variables, TAB and TAA, and the possible factors related to them was performed through Spearman correlation. The results obtained show that the factors associated with total attitude are educational stage (ρ = .142, p = .01) and school area (ρ = .175; p <.00), which correlate weakly with TAB, and slightly more strongly with TAA (ρ = .43, p <.00, for educational stage; ρ = .22; p <.00 for school area). Regarding the gender factor, no correlation was obtained for TAB (ρ = .05; p = .38) or for TAA (ρ = .10; p = .06). The normality of the distribution of TAB and TAA was analyzed with respect to the three factors studied: educational stage, gender and school area. The Kolmogorov-Smirnov test was used, which showed that the distributions for educational stage were normal (p = .20), but not for gender (p = .04) and the school area (p <.01). Therefore, the mean contrasts used were an ANOVA in the case of educational stage, and a Mann-Whitney U-test for the study of gender and school area. The data obtained from each of the analyzed factors are detailed below.
284 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
Educational stage Factor
The results of the attitude means for the educational stages studied: primary, compulsory secondary education and high school, are shown in table V.
TABLE V. Mean and standard deviation values of the attitude towards school science, before and after intervention, of the educational stages studied. * The significance level of the hypothesis testing is included.
TAB TAA t df p*
STAGE Mean Sd Mean Sd Primary 30.62 5.94 33.91 4.41 -6.61 210 <.00 Secondary 28.71 5.62 28.89 5.47 -.13 66 .90 High School 29.91 3.55 29.96 4.37 -.04 22 .97
Analysis of the attitude towards science classes within the educational stages studied, before and after implementation, showed that the most positive attitude was found in the primary and less positive stage in the compulsory secondary stage. On the other hand, when comparing the means, the ANOVA indicated that there was a difference regarding educational stage (F = 3.067, p = .048) in attitude before, TAB, and also in attitude after (F = 44.963; p <.00), TAA. When comparing the differences per student, the analysis shows that the differences for each stage are different (F = 5.709, p <.00). However, when we compared the attitude before and after the intervention of promoting scientific vocations for the same stage, TAA- TAB, we found that in the primary stage, there was a statistically significant difference of +3.29. In the other two stages analyzed the difference was .13 and .04, almost negligible, and had no statistical significance. Therefore, we can say that within the primary educational stage, this intervention has a significant positive effect on students’ attitude towards school science, but we cannot affirm that there is a similar effect for compulsory secondary education and high school.
Revista de Educación, 381. July-September 2018, pp. 275-295 285 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
Gender Factor
The values of overall attitude for both genders are shown in Table VI.
TABLE VI. Mean and standard deviation values before and after the intervention regarding gen- der.* The significance level of the hypothesis testing is included.
Gender Mean Sd Mann- Whitney U* Girls 30.45 5.87 TAB p=.38 Boys 29.84 5.64 Girls 32.21 4.87 TAA p=.06 Boys 32.82 5.40
The association between TAB and TAA and gender is not given in this data set, although the attitude after intervention could be considered marginally significant. A deeper analysis of gender could be made if a difference was found within any of the educational stages considered, as gender and educational stage factors are associated with each other (χ2 (N = 380, 2) = 6.67, p = .04). The means are shown in table VII.
TABLE VII. Average attitude and standard deviation per educational stage, before and after intervention, regarding gender. Mean (standard deviation)
M(Sd) Primary Secondary High school Girls 30.97(6.07) 29.45(5.73) 29.11(3.30) OAB Boys 30.30(5.82) 27.42(5.27) 30.43(3.74) Girls 33.53(4.23) 29.43(5.26) 30.00(3.81) OAA Boys 34.28(5.82) 27.96(5.81) 29.93(4.84)
The means are not different for girls and boys in the primary stage before the intervention (U = 5432.00, p = .39), nor in secondary stage (U = 427.50, p = .31) or high school (U = 49.50, p = .40). Something similar occurs after the intervention in primary (U = 5317.50, p = .10) and secondary (U = 457.50, p = .36). However, in high school, we found a marginally significant difference in means (U = 352.50, p = .07), as it
286 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science? had occurred in the total sample. That slight difference implies a slightly more positive attitude in boys.
School Area Factor
The results regarding this factor are shown in table VIII. In both overall attitudes before, TAB and after, TAA, we analyzed different mean values for rural and urban areas. These differences have statistical significance in both cases. A more positive attitude is observed among students in rural areas than in urban ones on both occasions.
TABLE VIII. Average attitude and standard deviation, before and after, regarding the school area. * The significance level of the hypothesis testing is included.
School Area Mean Sd Mann-Whitney U* Rural 31.60 5.54 TAB p<.00 Urban 29.41 5.72 Rural 33.72 4.57 TAA p<.00 Urban 31.15 5.30
The mean obtained for TAB in rural areas is 31.60, and for TAA, 33.72, which represents an increase of 2.12. In urban areas the values for TAB and TAA are, respectively, 29.41 and 31.15, which also represents an increase of 1.74, somewhat lower than in rural areas.
Discussion
This study assesses the effect of an outreach program which has the main goal of developing experiments related to student attitude towards school science which are appropriate for the cognitive level of the participating students. These experiments are implemented by the students themselves. In the assessment of this intervention, factors such as educational stage, gender and school area (rural or urban) were taken into account. A cross- sectional study was conducted, employing a quantitative methodology
Revista de Educación, 381. July-September 2018, pp. 275-295 287 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
that used a brief questionnaire. The sample consisted of 380 primary education, compulsory secondary education and high school students from the participating centers. A quasi-experimental design was used, since a pretest and a posttest were administered. The results show: a more positive attitude in school towards school science in the posttest (t = 5.46, df = 299; p <.01), this increase being greater among the students in primary education (t = -6.61; df = 210; p <.00). In rural students the effect of the program generates a more notable increase (p <.00); whereas a marginally significant influence was found regarding the program’s effect on attitude when gender was considered at the high school level (p = .06). In this study, in general, a more positive attitude towards school science is observed after participation in the project. The results of attitude improvement through interventions inside and outside the school have been reported by other authors. The studies found that are more comparable to this one are those by Weinberg, Basiley Albright (2011), Cardona et al (2012), and Barmby et al (2008). All three propose interventions outside the school carried out by people other than the typical classroom teacher, and the authors study the effect on attitude. If we compare the type of intervention carried out, those that are most similar to ours are the first and the third studies. The difference between them is that, in the first one the motivation towards science and mathematics is evaluated, but not attitude towards school science, as it is in the third one. Therefore, we focus our attention on the latter because of its similarity to this research. As in the study presented here, the study published by Barmby et al (2008) presents itinerant experiments (Lab in a Lorry) developed by scientists with students, and their influence on attitude towards school science is evaluated. In that case, they found an improvement in attitude, in line with the results presented here. However, both studies differ in that the latter worked only with secondary school students, while in the one we conducted, data is reported at the primary school level as well as for compulsory secondary education and high school. In addition, here we found a greater improvement in attitude among students in primary education. Therefore, the data referring to the primary stage and the improvement obtained through this intervention, are what set this study apart from others, given the absence of data related to interventions that involve experimentation with students at this educational level. The differences observed in the different stages in which the type of intervention does not equally affect the different
288 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science? groups could be justified either by the degree of participation of the students, by the adjustment to their knowledge or previous expectations, or by any other variable not considered in this study. However, despite finding that the overall attitude towards school science is more positive after the intervention, particularly at the primary school level, we are aware that there are methodological limitations. The sample is not evenly distributed among the stages, and is not representative of the student population of the country. This was due to the unequal participation of the different centers which made up the sample composition for this project, and, therefore limits the implications of this study. Regarding gender, no influence is observed on attitude towards school science on a general level, neither before nor after the intervention. Nonetheless, a marginally significant difference in high school students is noticed. The results found in the literature in this regard are controversial. On the one hand, in studies such as those by Weinberg, Basiley Albright (2011), and the one conducted by (Fernández Cézar & Solano Pinto, 2017) no differences were reported. This contrasts with what is shown in the studies by Marbà-Tallada and Solsona-Pairó (2012) or Smith, Pasero and McKenna (2014), who report that girls present a more negative attitude towards science, and highlight the difference in scientific preferences between boys and girls, with the former showing greater inclination towards scientific tasks. On the other hand, despite the fact that in the study by Fernández Cézar and Solano Pinto (2017) there is no difference in gender, it is reported that among primary school students who show a more positive attitude towards school science, most are girls. The marginally significant difference found in this study among high school students is in line with the research of Marbà-Tallada and Solsona-Pairó (2012) and that of Smith, Pasero and McKenna (2014), since girls present with a more negative attitude and a lesser effect is noticed. Results found regarding primary and secondary education students were in line with the studies by Fernández Cézar and Solano Pinto (2017), and Weinberg, Basiley Albright (2011). These results could be explained due to the difference in age between the students. The lack of difference found among the younger students might be attributed to the fact that they have yet to associate the gender roles that socially link science to males. On the other hand, even though there has been a general decline of scientific vocations since the first decade of this century (Vázquez Alonso, & Manassero-Mas, 2008), currently the majority of students in
Revista de Educación, 381. July-September 2018, pp. 275-295 289 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
the Faculties of Biochemistry or Environmental Studies are female. This fact supports the positive attitude in girls. In technical careers, such as engineering, this does not occur. Therefore, in addition to the age difference among the students, we identified another limitation of this study, which comes from the fact that the instrument used to quantify the attitude towards school science does not allow one to differentiate between health sciences or technical sciences. This should be an aspect to explore in the future in order to analyze if there is gender bias in all or only some scientific disciplines. Regarding the area where the school is settled, rural or urban, there are few studies that analyze this aspect in a general manner. The one carried out by Chetcuti and Kioko (2012) shows that, in Kenya, girls from rural environments have more positive attitudes towards science than those from urban environments. This aspect has not been studied in contexts closer to ours. In this study, the results coincide with those of Chetcuti and Kioko (2012), as the students from rural areas have a more positive attitude than those from urban ones, both before and after participation in the project. In the study by Fernández Cézar and Solano Pinto (2017) it was also observed that students from rural areas have a more positive attitude towards school science. Given the diversity of extracurricular activities between rural and urban environments, outreach interventions such as this project are necessary to promote equal opportunities. More studies are needed in this line to bring light to the educational situation of students in rural environments in Spain and Europe. However, as indicated at the beginning of this discussion, given the non-representativeness of the sample, which is not weighted with regard to the rural / urban aspect as is the population within the country, further research is necessary to confirm if in rural areas students present a more positive attitude towards science in a generalized manner.
Conclusions: limitations and prospective recommendations
The authors conclude that the evaluation of experimentation interventions involving students with first line researchers has a positive influence on students’ attitude towards school science, and to a lesser extent, on students at the secondary school level (compulsory and high school). The biggest depletion is produced between primary and compulsory
290 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science? secondary education, both before and after the intervention. There are some recent studies which analyze both the students’ and teachers’ emotions in the latter stage (Cortés, 2015), emphasizing their influence on them in the educational itineraries followed by students. There was no association of TA based on gender found in general, but it is associated with the region or context where the school is settled, being more positive in rural areas. An improvement is observed in both geographical areas after the school’s participation in the project. Given the implications raised in this study, taking into account its limitations due to both the sample composition and the duration of the intervention, it would be necessary to further analyze attitude towards school science and the kind of interventions that could influence it, particularly with students in secondary education (compulsory and high school). This intervention has a positive characteristic because students are involved in the experimentation, which contributes to a more positive attitude towards school science. The negative aspect, or the weakest point, is the short duration of the intervention, which results in slight changes that fade over time. Because of this fact, it is less effective intervening at the secondary education level in general, where the minds of the students are stronger and less modifiable due to their age. Students at this level would need more sustained interventions in order to see a more positive outcome. Therefore, we believe that active experimentation interventions involving students are needed in order to raise the scores related to attitude towards school science in primary education and to keep them high throughout the rest of their educational stages. Interventions such as the ones presented in this study, which call for students to manipulate the experiments and not merely observe or come to a conclusion instead of merely seeing its effects, will have a greater impact on their future science studies and will never be forgotten. For this purpose, placing focus on the science teachers would be most effective as they are the ones who are in charge of instruction. Since the development of this outreach project we are observing that many teacher in the schools that we visited exhibit insecurities when using experiments as a fundamental part of their science teaching. Thus, it is necessary to design appropriate training programs along with their corresponding resources and materials, and make these programs available to teachers so that student directed experiments such as these become a fundamental part of instruction in the science classroom.
Revista de Educación, 381. July-September 2018, pp. 275-295 291 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
References
Abad, F.J., Olea, J., Ponsoda, V. & García, C. (2011). Medición en ciencias sociales y de la salud. Madrid: Síntesis. Aikenhead, G. S., & Ryan, A. G. (1992). The Development of a New Instrument:‘Views on Science—Technology—Society’(VOSTS). Science Education, 76(5), 477-491. Alonso, Á. V., & Mas, M. A. M. (2007). Los intereses curriculares en ciencia y tecnología de los estudiantes de secundaria. Universitat Illes Balears, Servei de Publicacions i Intercanvi Científic. Alonso, Á. V., & Mas, M. A. M. (2008). La elección de asignaturas de ciencias: análisis de los factores determinantes. Revista española de Pedagogía, 541-558. Alonso, Á. V., & Mas, M. A. M. (2009). La relevancia de la educación científica: actitudes y valores de los estudiantes relacionados con la ciencia y la tecnología. Enseñanza de las ciencias: revista de investigación y experiencias didácticas, 27(1), 33-48. Alsop, S. (Ed.). (2005). Beyond Cartesian Dualism: Encountering Affect in the Teaching and Learning of Science (Vol. 29). Springer Science & Business Media. Barmby, P., Kind, P. M., & Jones, K. (2008). Examining changing attitudes in secondary school science. International journal of science education, 30(8), 1075-1093. Bennássar, A., Vázquez, A., Manassero, M. A. y García-Carmona, A. (Coords.) (2010). Ciencia, tecnología y sociedad en Iberoamérica: una evaluación de la comprensión de la naturaleza de ciencia y tecnología. Madrid, España: Organización de Estados Iberoamericanos para la Educación, la Ciencia y la Cultura (OEI). Bethke Wendell, K., & Rogers, C. (2013). Engineering design-based science, science content performance, and science attitudes in elementary school. Journal of Engineering Education, 102(4), 513-540. Briñol, P., Horcajo, J., De la Corte, L., Valle, C., Gallardo, I. y Díaz, D. (2004). El efecto de la ambivalencia evaluativa sobre el cambio de actitudes. Psicothema, 16, 373-377. Cardona, L. Z., Angulo, F., Soto, C. A., Quintero, S. M., Ceballos, A. F., Delgado, E.,& Cifuentes, L. J. (2012). ¿Contribuyen los talleres en el Museo de Ciencias a fomentar actitudes hacia la conservación del ambiente?Enseñanza de las ciencias: revista de investigación y experiencias didácticas, 30(3), 53-70.
292 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
Chen, S. (2006). Development of an instrument to assess views on nature of science and attitudes toward teaching science (VOSE). Science Education, 90(5), 803-819. Chetcuti, D. A., & Kioko, B. (2012). Girls’ attitudes towards science in Kenya. International Journal of Science Education, 34(10), 1571- 1589. Cortés, A. B. B. (2015). Las emociones en la enseñanza y el aprendizaje de las ciencias en educación secundaria. Enseñanza de las ciencias: revista de investigación y experiencias didácticas, 33(3), 199-200. de Pro Bueno, A., & Pérez Manzano, A. (2014). Actitudes de los alumnos de Primaria y Secundaria ante la visión dicotómica de la Ciencia. Enseñanza de las Ciencias, 32(3), 111-132. Di Martino, P., & Sabena, C. (2010). Teachers’ beliefs: the problem of inconsistency with practice. In Proceedings of the XXXIV Conference of the International Group for the Psychology of Mathematics Education (Vol. 2, pp. 313-320). Di Martino, P. & Zan, R. (2010). Me and maths: towards a definition of attitude grounded on students’ narratives. Journal of Mathematics Teacher Education, 13(1), 27-48. Eiser, J.R. (1989). Psicología social. Actitudes, cognición y conducta. Madrid: Pirámide. Fernández Cézar, R., & Solano Pinto, N. (2017). Actitud hacia las clases de Ciencias Naturales en la educación primaria en España. Revista electrónica de investigación educativa, 19(4), 112-123. doi: https:// doi.org/10.24320/redie.2017.19.4.1393. Heckman, J., Stixrud, J. y Urzua, S. (2006). The effects of cognitive and non- cognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24 (3), 411-482. Hong, Z. R., Lin, H. S., Wang, H. H., Chen, H. T., & Yang, K. K. (2013). Promoting and scaffolding elementary school students’ attitudes toward science and argumentation through a science and society intervention. International Journal of Science Education, 35(10), 1625- 1648. Kennedy, J., Quinn, F., & Taylor, N. (2016). The school science attitude survey: a new instrument for measuring attitudes towards school science. International Journal of Research & Method in Education, 39(4), 422-445. doi: 10.1080/1743727X.2016.1160046.
Revista de Educación, 381. July-September 2018, pp. 275-295 293 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto-Solano, N., Muñiz-Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
Manassero, M. A., Vázquez, A., & Acevedo, J. A. (2003). Cuestionario de opiniones sobre ciencia, tecnología y sociedad (COCTS). Princenton, NJ: Educational Testing Service. Información en línea en http://www. ets. org/testcoll. Manassero-Mas, M. A., Vázquez-Alonso, Á., & Acevedo-Díaz, J. A. (2001). Cuestionario de Opiniones Sobre Ciencia Tecnologia Y Sociedad (COCTS). Universidad de las Islas Baleares, Departamento de Psicologia. Marbà-Tallada, A. & Márquez Bargalló, C., (2010) ¿Qué opinan los estudiantes de las clases de ciencias? Un estudio transversal de sexto de Primaria a cuarto de ESO. Enseñanza de las Ciencias, 28(1), 19–30. Marbà-Tallada, A., & Solsona-Pairó, N. (2012). Identificación e interpretación de las posibles desigualdades formativas en ciencias de chicos y chicas en la educación obligatoria y el bachillerato. Cultura y Educación, 24(3), 289-303. Si hay diferencia. Mellado, V., Blanco, L. J., Borrachero, A. B., & Cárdenas, J. A. (2013). Las emociones en la enseñanza y el aprendizaje de las ciencias y las matemáticas. Ed: DEPROFE. Badajoz, España. Potvin, P. & Hasni, A. (2014). Interest, motivation and attitude towards science and technology at K-12 levels: a systematic review of 12 years of educational research. Studies in Science Education, 50 (1) 85–129, http://dx.doi.org/10.1080/03057267.2014.881626. Osborne, J., Simon, S. y Collins, S. (2003). Attitudes towards science: a review of the literature and its implications. International Journal of Science Education, 25(9), 1049 – 1079. ROSE, A. C. D. P. (2005). Proyecto ROSE: relevancia de la educación científica. Revista Eureka sobre enseñanza y divulgación de las ciencias, 2(3), 440-447. Rubba, P. A., & Harkness, W. L. (1993). Examination of preservice and in- service secondary science teachers’ beliefs about science-technology- society interactions. Science Education, 77(4), 407-431. Saleh, I. M., & Khine, M. S. (Eds.). (2011). Attitude research in science education: Classic and contemporary measurements. IAP. Schreiner, C. & Sjøberg, S. (2004). Showing the seeds of ROSE. Background, Rationale, Questionnaire Development and Data Collection for ROSE (The Relevance of Science Education) - a comparative study of students’ views of science and science education. Acta Didactica, (4/2004). Published: Dept. of Teacher Education and School Development, University of Oslo, Norway. Retrieved on 2/12/2014 from http://www. ils.uio.no/forskning/rose/documents/AD0404.pdf.
294 Revista de Educación, 381. July-September 2018, pp. 275-295 Received: 30-11-2017 Accepted: 23-04-2018 Fernández-Cézar, R., Pinto Solano, N., Muñiz Hernández, M. Do experimentation outreach programs improve attitudes towards school science?
Siegel, M. A., & Ranney, M. A. (2003). Developing the changes in attitude about the relevance of science (CARS) questionnaire and assessing two high school science classes. Journal of Research in Science Teaching, 40(8), 757-775. Smith, T. J., Pasero, S. L., & McKenna, C. M. (2014). Gender effects on student attitude toward science. Bulletin of Science, Technology & Society, 34(1-2), 7-12. Unal, A., & Altiparmak, M. (2013). The Effects of Webquests on Elementary School Seventh Grade Students’ Science Achievement, Attitude towards Science and Attitude towards Web Based Education while Teaching Science Subjects. In EDULEARN13 Proceedings (pp. 4165- 4173). IATED. Vázquez, A. (2013). La educación científica y los factores afectivos relacionados con la ciencia y la tecnología. En Las emociones en la enseñanza y el aprendizaje de las ciencias y las matemáticas, (pp.245- 278). Ed: DEPROFE. Badajoz, España. Vázquez Alonso, Á., Acevedo Díaz, J. A., Manassero Mas, M. A., & Acevedo Romero, P. (2006). Actitudes del alumnado sobre ciencia, tecnología y sociedad, evaluadas con un modelo de respuesta múltiple. Revista Electrónica de Investigación Educativa, 8(2), 1-37. Vázquez, Á., & Manassero, M. A. (2008). El declive de las actitudes hacia la ciencia de los estudiantes: un indicador inquietante para la educación científica. Revista Eureka sobre Enseñanza y Divulgación de las ciencias, 5(3). Vázquez- Alonso, A. & Manassero- Mas, M.A. (2011). El descenso de las actitudes en la educación obligatoria. Ciência & Educaçao, 17(2), 249- 268. Vázquez- Bernal, B. y Jiménez- Pérez, R. (2016). Modelización de un constructo teórico sobre la percepción del alumnado en procesos de indagación en ciencias. Revista de Psicodidáctica, 21(1), 25-44. Weinberg, A. E., Basile, C. G., & Albright, L. (2011). The effect of an experiential learning program on middle school students’ motivation toward mathematics and science. RMLE Online, 35(3), 1-12.
Contact address: Raquel Fernández-Cézar. Universidad de Castilla la Mancha, Fac. de Educación de Toledo, Dpto. de Matemáticas. Área de Didáctica. Av. Carlos III s/n, 45071, Toledo, Spain. E-mail: [email protected]
Revista de Educación, 381. July-September 2018, pp. 275-295 295 Received: 30-11-2017 Accepted: 23-04-2018
Reviews
Reviews
Pérez Pérez, C. (2016). Educación en valores para la ciudadanía. Estrategias y técnicas de aprendizaje. Bilbao: Desclée de Brouwer.
To the extent that the human being is released from instinct, he/she has moral ability and the possibility of making him/herself through education. For this reason, civic and citizenship values constitute a fundamental content without which education would not be possible nowadays. In the piece reviewed, Professor Cruz Pérez presents citizenship as a multisided concept that encompasses different spheres such as legal, social, economic, civic, intercultural or cosmopolitan; each with its educational implications, amplified during the last decades because of the growing interest on the subject among the European Union context. If we focus the scope of the Formal Education System as the author points out, teachers find it difficult to approach to values education. Among complexities remarked there are: lack of tradition to program teaching-contents; difficulty of using strategies for value and attitudes modification; complexity for introducing the social and familiar network organizations in school; indoctrination danger; the influence of Social Media; a strong disposition of many teachers to teach individually; and their formative necessities to exercise a quality values education. Anyway, the key question raised is “educating for what values?”. Answering to this question, the book makes a clear commitment to education for a minimal ethics, in tune with the proposals of relevant authors such as A. Cortina or J. Escámez. Under this concept various proposals have been gathered based on the idea that inside an open and plural society it is necessary and unavoidable to educate for values that can be assumed by everyone. To address these key issues, the book describes the main theories from which it is possible to educate in values: sociologism, clarification of values, development of moral judgment, character education and
Revista de Educación, 381. July-September 2018, pp. 299-302 299 philosophy for children; while presenting those pedagogies postulates derived from each one of them. Values education for exercising citizenship holds a complete chapter, where it is studied the historical trajectory, globalization and emergence of contexts full of cultural plurality, those legal aspects as normative rights, the European context and the current information and communication society. Strategies the author considers more suitable to educate citizens in the school context are collected, among which it should be highlighted the organization of centers according to a model of democratic participation, education for coexistence and conflict resolution, as well as collaboration and solidarity culture regarding the needs of the community, among others. A chapter is also dedicated to describing and analyzing techniques for learning values that have been most effective for training good citizens. To this goal, sixteen specific techniques are selected, describing in detail the aims, content, application, teacher’s role and evaluation methods. The objective is to provide educational intervention tools to teachers in an area where there it is a great lack of resources and information. Techniques usually finish with debate and exchange of impressions that have appeared throughout the activity, giving a conclusive character to the work done. Eventually, it is about bringing values to action and reflection in a constantly propositional and shared way. The book has its final touch regarding the complex subject of values evaluation, understood as continuous, global, formative and processual; becoming particularly relevant elements such as distinction between personal opinion and knowledge and skills of students, perception of comments, acquisition of moral values and the use of different types of evaluation instruments and techniques. It is, in short, a book that collects the theoretical updates without renouncing the more classic approaches nor stop providing practical solutions for values education, both from a school perspective and from a growing social and citizen concern.
Fran J. Garcia-Garcia
300 Revista de Educación, 381. July-September 2018, pp. 299-302 Monarca, H. y Thoilliez, B. (Coords.) (2017). La profesionalización docente: debates y propuestas. Madrid: Síntesis. pp. 146. ISBN: 978- 84-9171-045-5
Along its ten chapters, the authors bring the current worry of «filling the classrooms with “effective teachers”» (p. 9), trying to raise awareness of the present weakening of the teaching profession by endorsing a revised literature, at the same time that the importance of the initial training of teachers is claimed. Enric Prats and Ana Marín view the university as the main trainer of the future teachers, as well as they invite us to think about its present role, insisting on the necessity of providing a solid education based on a triple training dimension: cultural, intellectual and researching. Paul Standish criticizes the idea of education as an exchange within a closed economy, where education generates outcomes being at the service of accountability in the teaching practice. Thereby, he upholds «economies of excess of otherness and intensity» (p.40) to understand what does really matter. Then, Héctor Monarca develops the idea of teacher professionalism and introduces some fragmentations and contradictions. Highlighting its polysemic character, the author claims that alternative ways leading to «transforming and liberating praxis» (p.50) are possible. Bianca Thoilliez defends two main ideas: on the one hand, that orientating the teaching profession to produce objective learning outcomes fractures the basis of teaching; and on the other hand, that the best way to prepare the future teachers is not settled in a teacher professional development (TPD) model based on evidences, as educate is «more a heroic surgery than a technical protocol» (p.62). Afterwards, Geo Saura and Noelia Fernández-González develop the ideas of “teachers governed by numbers” and contemporary educational reforms based on commercial logics, showing the TPD as a new pretended professionalism where governments are modelling the profession. In the end, the authors disapprove the appearance of endoprivatization mechanisms that are globally de-professionalising teaching. Further, Fernando Gil Cantero refers to teachers as people who must cultivate themselves and worry about their own personal and intellectual growing, in other words, who want to be better. Therefore, he demands a pedagogical theory mastery and more reflection and educational judgment to give consistency to the own ideals and pedagogical convictions.
Revista de Educación, 381. July-September 2018, pp. 299-302 301 To conceptualize teacher’s identity is Tania Alonso’s task, who views initial training as a key moment to settle the basis of professional teacher’s identity and to overcome the identity crisis or «the lack of teacher’s own voice when it comes to telling themselves who are they and which role do they play in schools and society» (p.93). Endorsing Charles Taylor literature, she claims the need for a solid teacher’s identity. Jesús Manso compares the importance of the initial training of teachers for the TPD with the first years of life for people development. He defines the induction period to the teaching profession and reflects on the different ruptures that emerge within the process. To conclude, he requests more prominence of teachers in the actions being taking around these debates. David Reyero reminds us that «it is impossible to be a good teacher without mastering what is taught» (p.125) and demands a solid curricular training. Moreover, he criticizes the secondary role that cultural contents are taking and suggests increasing its value in the educational curriculum, examining the pedagogical competences, recruiting the best teachers and educating them in those cultural contents. Finally, Inmaculada Egido analyses the teaching practicum after narratively revising different studies. Focused in the national sphere, she explains the main points in relation to the roles and tasks accomplish by student teachers, the practicum tutors and supervisors, the schools, the relation between those and the university, and the organization, development and assessment of this period. The conclusions suggested, as she says, could be a first step for the future design of a quality practicum programme. The book presents a complete revision of the main points regarding the TPD, and along its pages the authors manage to catch our attention inviting us to take part of the educational debates presented and generating an inescapable self-reflection. Some of the authors demand more thinking, contact and discussion on the literature from the prospective teachers, ¿why not start by leaving this book on their tables? La profesionalización docente: debates y propuestas should be a must-read and a source of debate in the classrooms of the faculties of education.
Virginia Cabadas Maldonado
302 Revista de Educación, 381. July-September 2018, pp. 299-302