Chemistry Teacher International 2021; ▪▪▪(▪▪▪): 1–16

Research article

Antonios Bakolis, Dimitrios Stamovlasis and Georgios Tsaparlis* Explicit teaching of problem categorization using concept mapping, and an exploratory study of its effect on student achievement and on conceptual understanding – the case of chemical equilibrium problems https://doi.org/10.1515/cti-2019-0021 Received December 5, 2019; accepted January 25, 2021; published online February 18, 2021

Abstract: A crucial step in problem solving is the retrieval of already learned schemata from long-term memory, a process which may be facilitated by categorization of the problem. The way knowledge is organized affects its availability, and, at the same time, it constitutes the important difference between experts and novices. The present study employed concept maps in a novel way, as a categorization tool for chemical equilibrium problems. The objective was to determine whether providing specific practice in problem categorization improves student achievement in problem solving and in conceptual understanding. Two groups of eleventh-grade students from two special private seminars in Corfu island, Greece, wereused: the treatment group (N = 19) and the control group (N = 21). Results showed that the categorization helped students to improve their achievement, but the improvement was not always statistically significant. Students at lower (Piagetian) developmental level (in our sample, students at the transitional stage) had a larger improvement, which was statistically significant with a high effect size. Finally, Nakhleh’scatego- rization scheme, distinguishing algorithmic versus conceptual subproblems in the solution process, was studied. Dependency of problem solving on an organized knowledge base and the significance of concept mapping on student achievement were the conclusion.

Keywords: chemical equilibrium; concept mapping; Piagetian stages; problem categorization; problem solving

Introduction

As a goal of the teaching of chemistry is to foster meaningful learning, the teaching of strategies that cultivate complex and hierarchical cognitive structures can also facilitate development of cognitive abilities, such as problem solving, an everlasting topic in chemistry education. The way knowledge is organized in long-term memory is generally expected to relate to the degree of success in problem solving. According to cognitive

*Corresponding author: Georgios Tsaparlis, Department of Chemistry, University of Ioannina, GR-451 10 Ioannina, Greece, E-mail: [email protected]. https://orcid.org/0000-0002-0856-747X Antonios Bakolis, Hellenic Civil Aviation Authority, I. Kapodistrias Corfu Airport, GR-49100 Corfu, Greece, E-mail: [email protected] Dimitrios Stamovlasis, Department of Philosophy and Education, Aristotle University of Thessaloniki, GR-54124 Thessaloniki, Greece, E-mail: [email protected]. https://orcid.org/0000-0003-0808-9065

Open Access. © 2021 Antonios Bakolis et al., published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 International License. 2 A. Bakolis et al.: The case of chemical equilibrium problems

psychology, knowledge organization affects the availability and the retrieval process of conceptual schemata during problem solving. A previous study (Zikovelis & Tsaparlis, 2006) suggested and employed a linear categorization scheme for problems in the special topic of colligative properties of ideal solutions. By ‘linear’, we mean that the categories were listed one after the other. A treatment group showed a superior achievement that was not statistically significant. By dividing the students into high-, intermediate- and low-achievement subgroups on the basis of their performance in two highly reliable, nationally-examined upper secondary chemistry courses, it was found that students of the treatment intermediate subgroup outperformed in a statistically significant sense those of the control group, while no differentiation was found for the students of high and low achievement. The present study was designed to show whether an intervention-based problem-solving teaching approach affected student achievement. The research design contained the new tool of categorization through concept maps (Novak, 1985, 1990a, 1990b; Novak & Gowin, 1984). An interventional teaching with the use of this tool was conducted to investigate whether it would be possible to directly transfer the schemata used by expert solvers and whether they help beginners in both problem solving and conceptual understanding. The effect of Piagetian stages of cognitive development (Inhelder & Piaget, 1958) was examined, as well as the categorization scheme proposed by Nakhleh (1993), which distinguishes between algorithmic versus conceptual steps (or pieces or subproblems) in the solution process. At the outset, a fundamental distinction must be emphasized, that between problems and exercises (Johnstone, 1993). A real/novel problem requires that the solver must be able to use what has been termed as higher-order thinking or cognitive skills (HOTS or HOCS) (Zoller, 1993; Zoller & Tsaparlis, 1997). As a rule, extensive practice in problems in a particular area can turn problems into exercises, in which case, lower-order thinking/cognitive skills (LOTS or LOCS) suffice. For example, many problems in science can be solved by the application of well-defined procedures (algorithms) (Bodner, 1987) that can turn the problems into algorithmic exercises. Chemical equilibrium problems, despite the complexity of many of them, can become exercises with practice.

Rationale – knowledge organization and problem categorization

Problem categorization is part of knowledge organization. Knowledge that is well organized and connected in long-term memory is more easily recalled than specific information, which lacks organization and connections (Johnstone, 1991). Chi and Koeske (1983) studied the network representation of a four-year old child’s “dinosaur knowledge”, by comparing the structure of two mappings, based on three attributes: (1) number of links, (2) strength of links, and (3) the internal cohesion of the network in terms of higher-order groupings and specific patterns of interlinkages. It was found that the better structured set of dinosaurs was more easily remembered and retained by the child over a year, than the less structured set of dinosaurs. Chi, Feltovich, and Glaser (1981) investigated the representation of physics problems in relation to the physics knowledge in experts and novices, and found that experts and novices begin their problem representations with specifically different problem categories. Further, they were led to the conclusion that experts’ organization of a domain is according to the “deep structure” of the domain, whereas the schemes of novices are characterized by “surface” features. Based on the previous and similar studies, Smith (1992) used categorizations of genetics problems and reported that, as expected, faculty experts focused almost exclusively on conceptual principles, but students’ categorizations focused primarily on problem knowns and unknowns. De Jong and Ferguson-Hessler (1986) investigated whether good novice problem solvers have their knowledge arranged around problem types to a greater extent than poor problem solvers. In the subject of physics (electricity and magnetism), 12 problem types were categorized according to their underlying physics principles. Good novice problem solvers responded to problem types, but poor problem solvers seemed to be influenced to a greater extent by the surface characteristics. The conclusion was that an organization of knowledge around problem types might be highly conducive to good performance in problem solving by novice problem solvers. A. Bakolis et al.: The case of chemical equilibrium problems 3

Bunce, Gabel, and Samuel (1991) examined the effect of strategies on chemistry problem solving and supported Reif’s (1981, 1983) hypothesis that a key in improving the problem solving capacity is the employment of a solution strategy. Expert and novice solvers are quite different in this capacity: experts’ knowledge networks about chemical equilibrium are characterized by a multilevel structure that is constructed in such a way that they include possible solution methods, which are easily retrieved from long-term memory (Wilson, 1994). Also, from the field of management, Day and Lord (1992), reported that the results of a problem sorting task indicated that experts tended to categorize the ill-structured problems significantly faster than novices. Experts also had greater variance in the number of categories used and they incorporated more problem information. Bunce and Heikkinen (1986) have proposed the explicit method of problem solving (EMPS), which aims to teach novice students the problem-solving analysis procedures used by experts. The method is an organized problem-analysis approach and involves: a) decoding the information given in the problem (data, outcome/ goal), b) correlating the data in long-term memory (recall), and c) designing the solution before implementing a mathematical solution. According to Reif (1981), this analysis helps students encode the pertinent information of the problem, which is a major difference in the problem-solving behavior of experts and novices. Encoding is defined by Sternberg (1981) as the identification of each term in the problem and the retrieval from long-term memory of the attributes of these terms that are thought to be relevant to the solution of the problem. According to Bunce et al. (1991), an important part of the encoding process is problem categorization. If students cannot correctly categorize a problem, they will not be able to retrieve the relevant information from long-term memory. A subsequent step in EMPS leads students to relate the encoded parts of the problem in a schematic diagram of the solution path. In the case of a computational problem, students can use mathematics after such an analysis, to reach an algebraic solution and eventually a numerical answer. Problem categorization, aimed at improving students’ performance on chemistry problems, was studied by Bunce et al. (1991), on the basis of the EMPS. The authors reported that specific instruction in problem categorization techniques improved achievement scores for combination problems, requiring more than one chemical concept in their solution, but not for single-concept problems. On the other hand, such training alone was found insufficient to lead to conceptual understanding. Starting from the tenets that direct teaching of problem-solving methods to high school physics students meets with little success, as well as the dependency of problem solving on an organized knowledge base, Pankratius (1990) investigated the effect of concept mapping on upper-middle-class high school physics students’ science achievement, and concluded that mapping concepts prior to, during, and subsequent to instruction led to greater achievement. Concept mapping was found to be a key to organizing an effective knowledge base. Two papers by Wilson examined the network representation of knowledge about chemical equilibrium by means of concept maps. In the first paper (Wilson, 1994), it was found that the degree of hierarchical organization of conceptual knowledge (as demonstrated in concept maps constructed by the students) varied, and that the differences reflect achievement and relative experience in chemical equilibrium. Similar findings had previously been reported by Gussarsky and Gorodetsky (1988). In the second paper (Wilson, 1996), the relationships were examined between: (i) the structural characteristics of students’ concept maps about chemical equilibrium, and (ii) independent measures of their achievement in chemistry. The results revealed structural differences in conceptual organization about chemical equilibrium among students with different levels of achievement and thus relative expertise in the domain. The author concluded that the organization of declarative knowledge influences its accessibility in cognitive tasks. Finally, according to a recent study (Yuriev, Naidu, Schembri, & Short, 2017), chemistry problem solving depends, on the one hand, on chemical knowledge and, on the other hand, on the availability and deployment of what is termed as metacognitive problem-solving processes. Yuriev et al. employed the construct of scaffolding as a pedagogical process that enables a novice to complete a learning task that could not be accomplished unas- sisted. They scaffolded students’ metacognition, which involves declarative, procedural and conditional knowledge, as well as the steps that are necessary for successful problem solving: planning, organization of information for efficient processing, checking and correcting for mistakes, and evaluation of the solution 4 A. Bakolis et al.: The case of chemical equilibrium problems

(the result). Each of these steps can be guided through prompts or scaffolding. Knowledge organization, of which problem categorization is an integral part, is then a metacognitive process, which can be supported by scaffolding.

Research questions

1) Has the intervention using a categorization scheme with concept maps has had an effect on student performance in problem solving in chemistry equilibrium? 2) Has (Piagetian) developmental level of students played a role on student performance in connection with the presence or the lack of intervention? 3) Has the relevant conceptual learning of students been affected by the presence or the lack of the intervention and in connection with developmental level of students? 4) Has the application of algorithmic procedures by the students been affected by the presence or the lack of the intervention and in connection with developmental level of students? 5) Are there any patterns of change that characterize each of the four categories in the Nakhleh categorization according to the conceptual and the algorithmic nature of the steps (or pieces or subproblems) in the problem solving process?

Before proceeding, however, it is essential to review the education literature on the topic of chemical equilibrium. We will focus on studies on problem solving in chemical equilibrium, but will also consider some more general studies on chemical equilibrium.

Chemical equilibrium problems – review of the education literature

Chemical equilibrium problems are very important in the teaching of general and analytical chemistry. At the same time, they are most complex and difficult chemistry problems. It is not then surprising that researchers have studied these problems from several perspectives. In addition, many researchers have examined the chemical equilibrium concept itself (e.g. students’ understanding and misconceptions). Camacho and Good (1989) studied the problem-solving behaviors of experts and novices and reported many knowledge gaps and misconceptions about chemical equilibrium. A finding that is relevant to the present study is that by Gabel, Sherwood, and Enochs (1984), whose subjects used algorithmic methods without understanding the concepts upon which the problems were based. Niaz (1995) also compared student achievement on conceptual and computational problems of chemical equilibrium and reported that students who perform better on problems requiring conceptual understanding also perform significantly better on problems requiring manipulation of data, that is, computational problems. Voska and Heikkinen (2000) identified and quantified chemistry conceptions students use when solving chemical equilibrium problems requiring application of Le Chatelier’s principle. Using a 10-item, pencil and paper, two-tier diagnostic instrument, 11 prevalent incorrect student conceptions about chemical equilibrium were identified. Students consistently selected correct answers more frequently (53% of the time) than they provided correct reasons (33% of the time). Quílez and colleagues have produced many publications on various issues about chemical equilibrium. A main theme is Le Châtelier’s principle for which there are erroneous statements in textbooks regarding its generalizability and the limitations of its application in connection with a system at equilibrium (Quílez, 2004, 2019a; Quilez & Solaz, 1995). Quílez is in favor of avoiding the use of Le Châtelier’s principle and the employment instead of the construct of the reaction quotient (Q). In each case under consideration, the value of Q can be compared with the value of the chemical equilibrium constant, and in this way problems with Le Châtelier’s principle can be overcome. Although Q is introduced in several high school chemistry textbooks, it is seldom used for precise predictions regarding disturbances of the equilibrium. Solaz and Quílez discussed A. Bakolis et al.: The case of chemical equilibrium problems 5

the prediction of chemical equilibrium shift when changing the temperature at constant volume (Solaz and Quílez, 1998), and changes in the extent of reaction in open equilibria (Solaz and Quílez, 2001). Quílez-Díaz and Quílez-Pardo (2015a) analyzed general chemistry textbook errors in definitions, calculations and units of equilibrium constants. Quílez (2012) and Quílez-Díaz and Quílez-Pardo (2015b) discussed textbooks’ erroneous assumptions and how to overcome them with regard to quantities related to Gibbs energy as applied to chemical equilibrium and spontaneity. Finally, Quílez (2009, 2019b) discussed historical and epistemological issues that relate to the teaching of chemical equilibrium. Kousathana and Tsaparlis (2002) reported upper secondary students’ common misconceptions and errors in solving chemical equilibrium problems, while Tsaparlis, Kousathana, and Niaz (1998) examined the effect on upper-secondary student achievement of manipulating the logical structure as well as of the mental demand (M-demand) of chemical equilibrium problems, and reported that (Piagetian) developmental level played the dominant part. This was explained by taking into account that these problems were of an algorithmic nature for the subjects of that study, because of their extended practice (the ‘training effect’) with the problems. Demerouti, Kousathana, and Tsaparlis (2004a) identified 12th grade students’ misconceptions and conceptual difficulties with ionic equilibrium concepts and problems. In addition, the same authors (Demerouti, Kousathana, & Tsaparlis, 2004b) examined the effect of Piagetian developmental level (level of cognitive development or scientific reasoning) and of disembedding ability (field dependence/ independence) on 12th grade students’ conceptual understanding and problem-solving ability in the area of acid-base equilibria. It was found that both variables played an important role in the student performance, with disembedding ability clearly having the larger effect. Developmental level was connected with most cases of concept understanding and application, but less so with situations involving complex conceptual situations and/or chemical calculations. On the other hand, disembedding ability was involved both in situations that required conceptual understanding alone (especially in demanding cases), or in combination with chemical calculations. The effects of neo-Piagetian constructs have also been found to be predictive, but in a nonlinear manner, for students’ problem solving ability in non-algorithmic problems, such as organic synthesis (Stamovlasis and Tsaparlis, 2012). Ganaras, Dumon, and Larcher (2008) carried out an empirical study concerning the mastering of the chemical equilibrium concept by prospective physical sciences teachers. The results showed that the concept of chemical equilibrium had not acquired the status of an integrating and unifying concept for the majority of these prospective teachers, while experimental knowledge was only partly dissociated from theoretical knowledge. Ozmen (2008) determined prospective science student teachers’ alternative conceptions of the chemical equilibrium concept, by using a 13-item pencil and paper, two-tier multiple choice diagnostic instrument. The results showed that students did not acquire a satisfactory understanding of the concept. In addition, 17 alternative conceptions were identified, which were grouped under the headings of: the application of Le Châtelier’s principle; reliability of the equilibrium constant; heterogeneous equilibrium; and the effect of a catalyst. Cheung (2009) investigated secondary school chemistry teachers’ understanding of chemical equilibrium through interviews that asked them to predict what would happen to the equilibrium system N2(g) + 3H2(g) ⇋ 2NH3(g) if a small amount of nitrogen gas is added to the system at constant pressure and temperature. An over-emphasis of Le Châtelier’s principle was found to be a major contributing factor to teachers’ problem-solving failures. Teachers were classified into two problem-solving categories, on the basis of their cognitive sources of failures. Finally, we focus on some recent studies. Aydeniz and Dogan (2016) examined the impact of argumentation on pre-service science teachers’ conceptual understanding of chemical equilibrium. Akınand Uzuntiryaki-Kondakci (2018) examined the interactions among pedagogical content knowledge components of novice and experienced chemistry teachers in teaching reaction rate and chemical equilibrium topics in a qualitative multiple-case design study. Finally, Ollino et al. (2018) presented a method for teaching chemical equilibrium, in which students engage in self-learning mediated by the use of a new multimedia animation. Apre-testandpost-testofstudents’ understanding of the basic concepts of chemical equilibrium were conducted and concluded that those students with little or no previous knowledge acquired a better understanding of chemical equilibrium. In addition, 80% of the teachers agreed that the multimedia resource and its complementary activities had positive effects on students’ understanding. 6 A. Bakolis et al.: The case of chemical equilibrium problems

Method

Subjects

The present study was conducted with 11th grade students from a special private preparation seminar.1 The students followed the so-called “positive stream”, which prepared students to enter science, engineering, medicine, and agro-science tertiary departments, in two preparation seminars in Corfu island, Greece. Forty students were divided into two groups, treatment (N = 19, of which 8 males and 11 females) and control (N = 21, of which 13 males and 8 females).

Measures and procedures

1) The Piagetian level of cognitive development/developmental level of students was assessed by means of Lawson’s pencil-and paper Test of Formal Reasoning (Lawson, 1978), in its two-tier, multiple-choice revised form (Test of Scientiﬁc Reasoning) (Lawson, 1993). This test has been used by various researchers in science education (e.g. Lawson, 1983; Niaz, 1991; Tsaparlis, 2005; Tsaparlis et al., 1998). From the 13-item multiple-choice test, items 7, 8 and 13 were deleted from the analysis because they were not parallel to items on the original Lawson’s (1978) and similar tests of intellectual development (Westbrook & Rogers, 1994). One point was given for each item for which a correct choice was made, for both the basic question and the explanation. Students with marks between 4 and 6 were assigned as being in the transitional level, while students with marks between 7 and 10 were assigned as formal. Students at the concrete stage (marks between 0 and 3) were not detected in our sample. This scoring scheme was consistent with Lawson’s (1978) categorization. These data are reported as percentages in the present paper. 2) An additional feature of our study was the categorization of our students according to Nakhleh (1993), on the basis of four possible combinations of responding to each pair of algorithmic/conceptual questions: both conceptual and algorithmic correct (or high conceptual − high algorithmic): (1, 1); conceptual correct but algorithmic wrong (or high conceptual − low algorithmic: (1, −1); conceptual wrong but algorithmic correct (or low conceptual − high algorithmic): (−1, 1); and both conceptual and algorithmic wrong (or low conceptual− low algorithmic: (−1, −1). In this work, the steps in the solution of the used problems were distinguished into conceptual and algorithmic (see Appendix 2), and the performance of the students in the corresponding steps was used to identify the category to which each student was assigned. 3) The following two problems were used in a pre- and post-test design.

The problems

Both problems were entirely identical with regard to the structure and the questions. They differed in the chemical reaction involved and in the numerical data. The first problem (pre-test) involved the equilibrium H2 (g) + I2 (g) ⇋ 2HI (g), while the second problem

(post-test) involved the equilibrium N2 (g) + O2 (g) ⇋ 2NO (g). The post-test problem is reproduced below:

Post-test problem. A mixture consisting of 1 mol N2(g), 1 mol O2 (g) and 2 mol NO (g) is at equilibrium in a 1 L volume vessel, at a temperature of 0 °C, according to the equation:

N2 (g) + O2 (g) ⇋ 2NO (g)

(a) How many moles of NO must be added to the vessel at a temperature of 0 °C to obtain 2 mol N2 in the new equilibrium?

(b) A mixture consisting of 4 mol N2 and 4 mol O2 is introduced into a 2 L empty vessel at temperature 0 °C. Find the composition of the mixture in the vessel when chemical equilibrium is established.

Students’ responses were scored based on the basis of the solution steps/pieces/subproblems and the scoring scheme devised by one researcher, which was subsequently checked by a second marker (see Appendix 2). The correlation coefﬁcient between the two markings was found to be 0.94, so both the scoring scheme and the scoring itself are considered highly reliable. Besides the total score, two sub-scores (C) and (A) are recorded, which corresponds to conceptual and algorithmic questions respectively.

Instructional methodology

In the first teaching period, Lawson’s test was administered to all students. Following that, the subject of chemical equilibrium was taught for three teaching periods (of 50 min each), and then the teacher solved model problems for two periods. All students were taught by the same teacher (AB). At the end of this teaching, the initial (ﬁrst) test was administered to all students. In the remaining time, teaching was differentiated between the two groups. A. Bakolis et al.: The case of chemical equilibrium problems 7

Control group (CG): A two-period intensive problem solving session followed in a traditional teacher-centered mode. No mention of problem categorization was made to the CG, that is, the students had to construct themselves in their minds any categorization scheme through the practice in problem solving that was provided by the teacher. Treatment group (TG): A two-period teaching included supplying the concept maps to the students in ﬁnal form by the teacher. Examples of concept maps used are given in Appendix 1. Problem solving was also carried out in a traditional teacher-centered mode. Subsequently, the second test with the second chemical equilibrium problem was administered to all students.

Statistical analysis

Students’ achievement was explored considering the effect of the following independents variables: 1) The method of teaching (TG v. CG); 2) The Piagetian stage of cognitive development (developmental level). 3) The student classiﬁcation according to Nakhleh’s scheme.

To test for statistically significant differences in mean achievements, we employed the t-test for (according to the case) dependent or independent samples, calculating also the effect size by means of Cohen’s d (Cohen & Holliday, 1982; Kelley & Preacher, 2012). Because of the limited samples, bootstrapping estimates were made. The procedure involved sampling with replacement from the original data (Efron & Tibshirani, 1993; Simon, 1997). This reinforces the statistical significance of the findings and conclusions. The equivalence of the two groups was demonstrated by the statistically insignificant differences in students’ academic achievement and on their scores in Lawson’s test of cognitive development. The bootstrapped t-test retained the null hypothesis for the equivalence regarding the chemistry pre-test [Me = 75.25, sd = 22.20; Mc = 72.38, sd = 21.89; p > 0.05] and the

Lawson test [Me = 72.63, sd = 20.23; Mc = 69.04, sd = 19.21; p > 0.05]. Moreover, to avoid the inﬂuence of prior knowledge of students, we conducted the research before teaching chemical equilibrium in the school or in the private seminar.

Results

Comparison of overall achievement in the problems

Figure 1 shows the overall percentage achievement (achievement in the whole test) in pre-test and post-test for the CG (left) and the TG (right). We observe that for both groups there is an increase in mean achievement from pre-test to post-test, but the difference is statistically signiﬁcant only for the TG t = 4.38, p < 0.001. Further, for this case, Cohen’s effect size value (d = 1.00) suggests a high practical signiﬁcance (d > 0.8).

Effect of Piagetian developmental level

Figure 2 shows overall percentage achievement in pre-test and post-test for transitional and formal students of the CG (left) and the TG (right). In the CG there were 12 formal and 9 transitional students. In the TG there were 12

Figure 1: Overall percentage achievement in pre-test and post-test for the control group (left) and the treatment group (right). 8 A. Bakolis et al.: The case of chemical equilibrium problems

Figure 2: Overall percentage achievement in pre-test and post-test for transitional and formal students of the control group (left) and the treatment group (right). formal and 7 transitional students. Regarding the change from pre-to post-test, we observe that achievement has improved but only for transitional students of the TG it is statistically signiﬁcant: t = 7.48, p < 0.001. Further, for this case, Cohen’s effect size value (d = 2.64) suggests a high practical signiﬁcance. Figure 3 shows the pattern of change of overall percentage achievement for transitional (left) and formal students (right) of the control group and the treatment group.

Conceptual understanding

Figure 4 shows the CG’s (left) and the TG’s (right) change in transitional and formal students’ achievement in the conceptual steps of the problems. For the CG, we observe that there are changes (improvements) from the pre-to the post-test, but they are not statistically significant. For the TG, we observe that there is a statistically significant improvement from the pre-to the post-test for transitional students, but not for formal students: for transitional students, t = 7.00, p < = 0.001. Further, for this case, Cohen’s effect size value (d = 2.48) suggests a high practical significance. Figure 5 shows the pattern of change of conceptual understanding for transitional (left) and formal students (right).

Application of algorithmic procedures

Figure 6 shows the CG’s (left) and the TG’s (right) change in transitional and formal students’ percentage achievement in the algorithmic steps of the problems. We observe that for both groups there are very small increases in achievement from pre-test to post-test, but the differences are not statistically signiﬁcant. Figure 7 shows the pattern of change of application of algorithms for transitional (left) and formal students (right).

Figure 3: Pattern of change of overall percentage achievement for transitional (left) and formal students (right) of the control group and the treatment group. A. Bakolis et al.: The case of chemical equilibrium problems 9

Figure 4: Achievement in conceptual understanding in pre-test and post-test for transitional and formal students of the control group (left) and the treatment group (right).

Figure 5: Pattern of change of conceptual understanding for transitional (left) and formal students (right) of the control group and the treatment group. Max mark = 100.

Figure 6: Achievement in algorithmic procedures in pre-test and post-test for transitional and formal students of the control group (left) and the treatment group (right). Max mark = 100.

Figure 7: Pattern of change of algorithmic procedures for transitional (left) and formal students (right) of the control group and the treatment group. Max mark = 100. 10 A. Bakolis et al.: The case of chemical equilibrium problems

Achievement according to the four Nakhleh categories

Figure 8 shows the overall achievements according to the four Nakhleh categories for the CG (left) and the TG (right). In the CG there were 5, 5, 2, and 9 students in the categories (−1, −1), (−1, 1), (1, −1), and (1, 1) respectively. In the TG these figures were: 3, 4, 3, and 9 respectively. The diagram for the CG shows no important changes. On the other hand, for TG we find statistically significant differences (p < 0.001) for three of the categories: high conceptual−low algorithmic (1, −1), low conceptual−high algorithmic (−1, 1), and low conceptual − low algorithmic (−1, −1). Further, for the TG, Cohen’seffectsizevalues(d = 1.41, 2.64 and 2.27 respectively) suggest high practical significances with regard the paired-samples t tests for the (1, −1), (−1, 1) and (−1, −1) categories. The high conceptual−high algorithmic category (1, 1) showed no improvement. Figure 9 shows the patterns of change from pre-test (left) to post-test (right) for each Nakhleh category. While in the pre-test (left) the two groups, CG and TG, have no statistical differences across the Nakhleh’s categorization scheme, in post-test (right) the Nakhleh’s categories high conceptual−low algorithmic (1, −1) and low conceptual−high algorithmic (−1, 1) showed significant statistical differences (p < 0.001). Further, for the post-test, Cohen’s effect size values (d = 1.99 and 1.79 respectively) suggest high practical significances with regard the independent samples t-tests for the (1, −1) and (−1, 1) categories.

Figure 8: Overall percentage achievement in pre-test and post-test according to the four Nakhleh categories for the control group (left) and the treatment group (right).

Figure 9: Comparative percentage achievements for control ad treatment groups within Nakhleh’s categorization scheme for pre- test) (left) and post-test (right).

Figure 10: Achievement in the constituent schemas and part steps of the problems for the CG (left) and the TG (right). A. Bakolis et al.: The case of chemical equilibrium problems 11

The part steps in the problem solution process

The achievement in the separate schemas and the individual solution steps (or pieces or subproblems)that required algorithmic procedures and those that required conceptual understanding were compared for the CG and the TG and the results are shown in Figure 10. Difficult proved steps A1b, A1c, A3a, A3b, A3c, B2b and B2c (all these steps were algorithmic), with most difficult A3c, and B2c, which were the final computational steps. On the other hand, easy proved steps A1a, A2a, A2b and B1a, all conceptual. The most spectacular improvement from pre-to post test was in steps A2a and A2b for the TG only. This is very interesting because these constitute the steps in schema A2: Disturbance of equilibrium and establishment of a new equilibrium.

Conclusions

The analysis of the data demonstrated that students in the treatment group (TG) who were taught problem categorization using concept maps as the categorization tool achieved a better overall score after the intervention (Research Question 1). The ranking of students in stages of Piagetian cognitive development showed that the scores of transitional students, especially in those parts of the solution that require conceptual understanding, were further improved, with a large effect size (Research Questions 2 and 3). In contrast, the formal students did not show any improvement. In addition, with regard to the answers to Research Questions 4 and 5, the separation of students according to the four Nakhleh categories (based on the distinction between the conceptual understanding and the algorithmic dimension) showed that the scores in categories high conceptual-low algorithmic and low conceptual-high algorithmic were improved, with a large effect size, especially in those questions, which require conceptual understanding. On the contrary, the algorithmic capacity did not show any statistically significant increases. In the case of the category high conceptual-high algorithmic (as was the case with the formal students), the achievement was quite high and remained high. It is therefore obvious that teaching problem categorization using concept maps, especially in the limited time available for teaching school chemistry, can positively affect at least students with lower cognitive abilities. Besides the effect of explicit teaching and the use of concept mapping, the present explorative study supports the thesis that algorithmic problem solving and conceptual understanding are two distinct dimensions in learning sciences. They should be distinguished as latent variables in psychological measurements and treated differently in teaching interventions and curricula (Stamovlasis et al., 2004, 2005).

Endpoint

We referred in the introduction, on the one hand, to the recognition of the dependency of problem solving on an organized knowledge base, and, on the other hand, to the significant results of concept mapping on student achievement (Pankratius, 1990). In addition, the work of Bunce et al. (1991) showed that problem categorization helps students to retrieve the relevant information from long-term memory for the solution of the problem. In modern terms, problem categorization is a metacognitive process, which can be supported by scaffolding/ prompts (Yuriev et al., 2017). This scaffolding involves the steps for successful problem solving, among which the organization of information for efﬁcient processing. Despite the limitation of the small samples of the present study, its findings are in line with the above assertions. They also should be combined with the results of the previous study (Zikovelis and Tsaparlis, 2006), which examined the effect of a linear categorization scheme for problems in colligative properties of ideal solutions: in that study, students were grouped into high-, intermediate- and low-achievement subgroups on 12 A. Bakolis et al.: The case of chemical equilibrium problems

the basis of their performance in two nationally-examined upper secondary chemistry courses, and it was found that students of the intermediate treatment subgroup outperformed in a statistically significant sense those of the control group, while no differentiation was found for the students of high and low achievement. The results of both studies showed the usefulness of problem categorization for intermediate subgroups of students. In the previous study, this intermediacy derived from subgroups intermediate in achievement in problem solving, while in the present study intermediacy concerned on the one hand the transitional students in the Piagetian sense, and on the other hand the categories high conceptual-low algorithmic and low conceptual-high algorithmic according to the Nakhleh categorization scheme. Students of intermediate achievement as well as at the transitional stage in the Piagetian sense are both capable and willing to be affected by the methodology. On the other hand, it may be the case that high-achieving students as well as formal students in the Piagetian sense are capable of constructing mentally the categorization schemes, and therefore are not in a need to receive special instruction, although the categorization scheme and the relevant concept mapping (especially if constructed by the students themselves - see below) could also be beneficial to them. Could anything said about concrete students, who were not represented in our sample? If we admit that concrete students are expected to be in general low achievers (but, depending on the task, by no means not all of them), the findings of the Zikovelis and Tsaparlis (2006) study point to a low or no benefit for them. There might be many possible reasons for such a failure: insufficient conceptual learning, deficiencies in the proper understanding of the problem statements, weakness in manipulating the mathematical relations entering the problems, lack of interest in the particular lesson and/or the whole school process. But all this is just spec- ulation because of lack of relevant data in this study. A final point that should be taken into account: in both the above studies (one on problems in colligative properties and another on problems in chemical equilibrium) the categorization schemes were provided to the students by the teachers in final form. It would be of great interest to consider the case where an active/ constructivist instructional method were to be employed, one in which students would be encouraged to construct themselves (on their own or collaboratively in groups, under the guidance of their teacher) the categorization scheme and the concept maps, of course under the guidance of their teachers. Further, ample practice in doing this will also develop both ability and confidence.

Note

(1) In Greece, private seminars (‘frontistiria’) run in parallel with the public school, that is secondary students attend at the same time both normal school and private seminar(s). Students’ parents pay for their chil- dren’s private seminars, where teaching takes place in small groups (ca. 10–15 students) and students pay special attention to be prepared for the higher-education entrance examinations.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission. Research funding: None declared. Conﬂict of interest statement: The authors declare no conﬂicts of interest regarding this article. A. Bakolis et al.: The case of chemical equilibrium problems 13

Appendix 1: Examples of the used concept maps.

Figure A1.1: General categories of chemical equilibrium problems.

Figure A1.2: Α concept map for chemical equilibrium problems.

Appendix 2: The solution process and the marking scheme for the post-test problem. 14 A. Bakolis et al.: The case of chemical equilibrium problems

Schemas and part steps (or pieces or subproblems) Step* Marks Type of step*** in the solution process assigned**

Schema A: The condition of initial chemical equilibrium ( marks)  KC = [NO] /[N][O]Aa  C [NO] =  mol/L, [N]= mol/L, [O]= mol/L Ab  A  Kc =  /( × )= Ac  A Schema A: Disturbance of equilibrium and establishment of a new equilibrium ( marks) When x mol NO are added to the reaction vessel, equilibrium shifts to Aa  C the left: suppose that y mol NO react. At the new equilibrium: ( + y/) mol N and also ( + y/) mol O, Ab  C while mol NO =  + x − y. According to the problem information: mol N at the new equilibrium =  =( + y/), hence y = . Therefore, at the new equilibrium there are  mol N,  mol O and x mol NO. Schema A. Numerical computations ( marks)  KC = [NO] /[N][O]= (KC is unchanged since there was no change of Aa  A temperature) At equilibrium: [NO] = x mol/L, [N]= mol/L, [O]= mol/L Ab  A  KC =  = x /( × ), hence x =  mol. (THIS IS THE REQUIRED OUTCOME Ac  A OF PART A OF THE PROBLEM) Schema B: The condition of new chemical equilibrium ( marks) Since only reactants are present, the reaction takes placed to the right, Ba  C until chemical equilibrium is established: z mol N react with z mole O, producing z mol NO. At equilibrium: z mol NO, ( − z) mol N and ( − z) mol O.Bb  A Schema B. Numerical computations ( marks)  KC = [NO] /[N][O]= (KC is unchanged since there is no change of Ba  A temperature) At equilibrium: [NO] = z mol/ L=z mol/L, [N]=( − z)/ mol/L, Bb  A [O]=( − z)/ mol/L. Hence:      KC =  = z /[( − z)/][( − z)/], →  = z /( − z) →  = z /( − z) → Bc  A  = z/( − z), hence z = . Therefore, the ﬁnal composition of the equilibrium mixture will be:  mol NO,  mol N and  mol O. (THIS IS THE REQUIRED OUTCOME OF PART B OF THE PROBLEM)

*The symbols for the steps used here (Aa, Bb, etc.) have nothing to do with similar symbols that are used in the concept maps of Appendix . **Total marks = . *** Total marks for conceptual steps (C) = ; total marks for algorithmic steps (A) = .

References

Akın, F. N., & Uzuntiryaki-Kondakci, E. (2018). The nature of the interplay among components of pedagogical content knowledge in reaction rate and chemical equilibrium topics of novice and experienced chemistry teachers. Chemistry Education Research and Practice, 19(1), 80–105. Aydeniz, M., & Dogan, A. (2016). Exploring the impact of argumentation on pre-service science teachers’ conceptual understanding of chemical equilibrium. Chemistry Education Research and Practice, 17(1), 111–119. Bodner, G. (1987). Role of algorithms in teaching problem solving. Journal of Chemical Education, 64(6), 513–514. Bunce, D. M., Gabel, D. L., & Samuel, J. V. (1991). Enhancing chemistry problem-solving achievement using problem categorization. Journal of Research in Science Teaching, 28(6), 505–521. Bunce, D. M., & Heikkinen, H. (1986). The effects of an explicit problem-solving approach on mathematical chemistry achievement. Journal of Research in Science Teaching, 23(1), 11–20. Camacho, M., & Good, R. (1989). Problem solving and chemical equilibrium: Successful versus unsuccessful performance. Journal of Research in Science Teaching, 26(3), 251–272. Cheung, D. (2009). Using think-aloud protocols to investigate secondary school chemistry teachers’ misconceptions about chemical equilibrium. Chemistry Education Research and Practice, 10(2), 97–108. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problem by experts and novices. Cognitive Science, 5(2), 121–152. A. Bakolis et al.: The case of chemical equilibrium problems 15

Chi, M. T. H., & Koeske, R. P. (1983). Network representation of a child’s dinosaur knowledge. Developmental Psychology, 19(1), 29–39. Cohen, L., & Holliday, M. (1982). Statistics for social scientists. London: Harper & Row. Day, D. V., & Lord, R. G. (1992). Expertise and problem categorization: The role of expert processing in organizational sense-making. Journal of Management Studies, 29(1), 35–47. De Jong, T., & Ferguson-Hessler, M. G. M. (1986). Cognitive structures of good and novice problem solvers in physics. Journal of Educational Psychology, 78(4), 279–288. Demerouti, M., Kousathana, M., & Tsaparlis, G. (2004a). Acid-base equilibria, Part I: Upper secondary students’ misconceptions and difficulties. The Chemical Educator, 9(2), 122–131. Demerouti, M., Kousathana, M., & Tsaparlis, G. (2004b). Acid-base equilibria, Part II: Effect of developmental level and disembedding ability on students’ conceptual understanding and problem-solving ability. The Chemical Educator, 9(2), 132–137. Efron, B., & Tibshirani, R. (1993). An introduction to the bootstrap. New York: Chapman & Hall. Gabel, D. L., Sherwood, R. D., & Enochs, L. (1984). Problem-solving skills of high school chemistry students. Journal of Research in Science Teaching, 21(2), 211–233. Ganaras, K., Dumon, A., & Larcher, C. (2008). Conceptual integration of chemical equilibrium by prospective physical sciences teachers. Chemistry Education Research and Practice, 9(3), 240–249. Gussarsky, E., & Gorodetsky, M. (1988). On the chemical equilibrium concept. Constrained word association and conception. Journal of Research in Science Teaching, 25(5), 319–333. Inhelder, B., & Piaget, J. (1958). The growth of logical thinking from childhood to adolescence. New York: Basic Books. Johnstone, A. H. (1991). Thinking about thinking. International Newsletter on Chemical Education, 36,7–11. Johnstone, A. H. (1993). Introduction. In C. Wood, & R. Sleet (Eds.), Creative problem solving in chemistry (pp. iv–vi). London: The Royal Society of Chemistry. Kellet, K., & Preacher, K. J. (2012). On effect size. Psychological Methods, 17(2), 137–152. Kousathana, M., & Tsaparlis, G. (2002). Students’ errors in solving numerical chemical-equilibrium problems. Chemistry Education Research and Practice, 3(1), 5–17. Lawson, A. E. (1978). Development and validation of the classroom test of formal reasoning. Journal of Research in Science Teaching, 15(1), 11–24. Lawson, A. E. (1983). Predicting science achievement: The role of developmental level, disembedding ability, mental capacity, prior knowledge, and beliefs. Journal of Research in Science Teaching, 20(2), 117–129. Lawson, A. E. (1993). Classroom test of scientific reasoning. Tempe, AZ: Arizona State University. Nakleh, M. B. (1993). Are our students’ conceptual thinkers or algorithmic problem solvers? Journal of Chemical Education, 70(1), 52–55. Niaz, M. (1991). Correlates of formal operational reasoning: A neo-Piagetian analysis. Journal of Research in Science Teaching, 28(1), 19–40. Niaz, M. (1995). Relationship between students performance on conceptual and computational problems of chemical equilibrium. International Journal of Science Education, 17(3), 343–355. Novak, J. D. (1985). Metalearning and metaknowledge strategies to help students learn how to learn. In Cognitive structure and conceptual change (pp. 101–115). Orlando, FL: Academic Press. Novak, J. D. (1990a). Concept mapping: A useful tool for science education. Journal of Research in Science Teaching, 27(10), 937–949. Novak, J. D. (1990b). Concept maps and Vee diagrams: Two metacognitive tools to facilitate meaningful learning. Instructional Science, 19(1), 29–52. Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. New York: Cambridge University Press. Ollino, M., Aldoney, J., Dominguez, A. M., & Merino, C. (2018). A new multimedia application for teaching and learning chemical equilibrium. Chemistry Education Research and Practice, 19(1), 364–374. Ozmen, H. (2008). Determination of students’ alternative conceptions about chemical equilibrium: A review of research and the case of Turkey. Chemistry Education Research and Practice, 9(3), 225–233. Pankratius, W. J. (1990). Building an organized knowledge base: Concept mapping and achievement in secondary school physics. Journal of Research in Science Teaching, 27(4), 315–333. Quílez, J. (2004). Changes in concentration and in partial pressure in chemical equilibria: Students’ and teachers’ misunderstandings. Chemistry Educastion Research and Practice, 5(3), 281–300. Quílez, J. (2009). From chemical forces to chemical rates: A historical/philosophical foundation for the teaching of chemical equilibrium. Science & Education, 18(9), 1203–1251. Quílez, J. (2012). First-year university chemistry textbooks’ misrepresentations of Gibbs energy. J. Chem. Educ., 89(1), 87–93. Quílez, J. (2019a). A categorisation of the terminological sources of student difficulties when learning chemistry. Studies in Science Education, 55(2), 121–167. Quílez, J. (2019b). A historical/epistemological account of the foundation of the key ideas supporting chemical equilibrium theory. Foundations of Chemistry, 21(2), 221–252. 16 A. Bakolis et al.: The case of chemical equilibrium problems

Quílez-Díaz, A., & Quílez-Pardo, J. (2015a). Avoiding general chemistry textbooks’ misrepresentations of chemical equilibrium constants. African Journal of Chemical Education, 5(2), 72–86. Quílez-Díaz, A., & Quílez-Pardo, J. (2015b). Avoiding first-year university chemistry textbooks’ misrepresentations in the teaching of spontaneous reactions. Química Nova, 38(1), 151–155. Quilez, J., & Solaz, J. J. (1995). Students’ and teachers’ misapplication of the Le Chȃtelier’s principle. Implications for teaching chemical equilibrium. Journal of Research in Science Teaching, 32(9), 939–958. Reif, F. (1981). Teaching problem solving – a scientific approach. The Physics Teacher, 19(5), 310–316. Reif, F. (1983). How can chemists teach problem-solving? Journal of Chemistry Education, 60(11), 949–953. Simon, L. J. (1997). Resampling: The new statistics. Arlington: Resampling Stats Inc. Smith, M. U. (1992). Expertise and the organization of knowledge: Unexpected differences among genetic counselors, faculty, and students on problem categorization tasks. Journal of Research in Science Teaching, 2(2), 179–205. Solaz, J. J., & Quílez, J. (1998). Unequivocal prediction of chemical equilibrium shift when changing the temperature at constant volume. Physics Education, 15(1), 29–33. Solaz, J. J., & Quílez, J. (2001). Changes in the extent of reaction in open equilibria. Chem. Educ. Res. Pract., 2(3), 303–312. Stamovlasis, D., & Tsaparlis, G. (2012). Applying catastrophe theory to an information-processing model of problem solving in science education. Science Education, 96(3), 392–410. Stamovlasis, D., Tsaparlis, G., Kamilatos, C., Zarotiadou, E., & Papaoikonomou, D. (2004). Conceptual understanding versus algorithmic problem solving: A principal component analysis of a national examination. The Chemical Educator, 9(6), 398–405. Stamovlasis, D., Tsaparlis, G., Kamilatos, C., Zarotiadou, E., & Papaoikonomou, D. (2005). Conceptual understanding versus algorithmic problem solving: Further evidence from a national examination. Chemistry Education Research and Practice, 6(2), 104–118. Sternberg, R. J. (1981). Intelligence and nonentrenchment. Journal of Educational Psychology, 73(1), 1–16. Tsaparlis, G. (2005). Non-algorithmic quantitative problem solving in university physical chemistry: A correlation study of the role of selective cognitive factors. Research in Science & Technological Education, 23(2), 125–148. Tsaparlis, G., Kousathana, M., & Niaz, M. (1998). Molecular-equilibrium problems: Manipulation of logical structure and of M- demand, and their effect on student performance. Science Education, 82(4), 437–454. Voska, K. W., & Heikkinen, H. W. (2000). Identification and analysis of student conceptions used to solve chemical equilibrium problems. Journal of Research in Science Teaching, 37(2), 160–176. Westbrook, S. L., & Rogers, L. N. (1994). Examining the development of scientific reasoning in ninth-grade physical science students. Journal of Research in Science Teaching, 31(1), 65–76. Wilson, J. M. (1994). Network representations of knowledge about chemical equilibrium: Variations with achievement. Journal of Research in Science Teaching, 31(10), 1133–1147. Wilson, J. M. (1996). Concept maps about chemical equilibrium and students’ achievement scores. Research in Science Education, 26(2), 169–185. Yuriev, E., Naidu, S., Schembri, L. S., & Short, J. L. (2017). Scaffolding the development of problem-solving skills in chemistry: Guiding novice students out of dead ends and false starts. Chemistry Education Research and Practice, 18(3), 486–504. Zikovelis, V., & Tsaparlis, G. (2006). Explicit teaching of problem categorization and a preliminary study of its effect on student performance – the case of problems in colligative properties of ideal solutions. Chemistry Education Research and Practice, 7(2), 114–130. Zoller, U. (1993). Are lecture and learning compatible? Maybe for LOCS: Unlikely for HOCS. Journal of Chemical Education, 70(3), 195–197. Zoller, U., & Tsaparlis, G. (1997). Higher and lower-order cognitive skills: The case of chemistry. Research in Science Education, 27(1), 117–130.