<<

Using Eye-Tracking to Assess the Application of Divisibility Rules when Dividing a Multi-Digit Dividend by a Single Digit Pieter Potgieter Pieter Blignaut Central University of Technology, Free State University of the Free State South Africa South Africa [email protected] [email protected]

ABSTRACT 1 INTRODUCTION The Department of Basic Education in South Africa has identified In 2013 and 2014, around 7 million learners of South Africa from certain problem areas in Mathematics of which the factorisation of Grade 1 to Grade 6 and Grade 9 took part in the Annual National numbers was specifically identified as a problem area for Grade 9 Assessments (ANA) [1, 2] Only 3% of Grade 9 learners achieved learners. The building blocks for factorisation should already have more than 50% in Mathematics. The Department of Basic Education been established in Grades 4, 5 and 6. Knowing the divisibility rules, [3, 4] has, through the ANAs, identified certain problem areas of will assist learners to simplify mathematical calculations such as which the factorisation of numbers was specifically identified as a factorisation of numbers, manipulating fractions and determining if problem area for Grade 9 learners. It is expected that the building a given number is a . When a learner has to indicate, blocks for factorisation should already have been established in by only giving the answer, if a dividend is divisible by a certain Grades 4, 5 and 6 and related questions appeared in the ANA papers single digit divisor, the teacher has no insight in the learner’s for these grades. reasoning. If the answer is correct, the teacher does not know if the Knowing the divisibility rules (Table 1) would enable learners to learner guessed the answer or applied the divisibility rule correctly quickly determine if a number, referred to as the dividend, is or incorrectly. divisible by a specific single digit divisor without having to do long A pre-post experiment design was used to investigate the effect calculations. Evidence of formal instructions to apply divisibility of revision on the difference in gaze behaviour of learners before rules for 2 to 12 was found in the workbook for Grade 5 to and after revision of divisibility rules. The gaze behaviour was Grade 7 learners [5-7], but despite this early exposure, the fact that analysed before they respond to a question on divisibility. Grade 9 learners lack the ability to do factorisation, is concerning. It is suggested that if teachers have access to learners’ answers, motivations and gaze behaviour, they can identify if learners (i) Table 1: Divisibility rules guessed the answers, (ii) applied the divisibility rules correctly, (iii) applied the divisibility rules correctly but made mental calculation Divisor Rule errors, or (iv) applied the divisibility rules wrongly. 2 The last digit must be even. 3 The sum of the digits must be divisible by 3. The number formed by the last two digits must be CCS CONCEPTS 4 divisible by 4. • Computing methodologies → Tracking; Activity recognition 5 The last digit must be 0 or 5. and understanding • Information systems → Relevance The number must be divisible by both 2 and 3 6 assessment • Social and professional topics → Student according to the above-mentioned rules. The number formed by the last three digits must be assessment 8 divisible by 8. KEYWORDS 9 The sum of all the digits must be divisible by 9.

Divisibility rules, eye-tracking, Mathematics The most basic formats of assessment consist of multiple choice ACM Reference format: and true/false questions [8]. When a learner has to indicate if a P.H. Potgieter and P.J. Blignaut. 2017. Using Eye-Tracking to Assess the dividend is divisible by a certain single digit divisor by only Application of Divisibility Rules when Dividing a Multi-Digit Dividend by a answering true or false, the teacher has no insight in the learner’s Single Digit Divisor. In Proceedings of SAICSIT ’17, Thaba Nchu, South Africa, reasoning. If the answer is correct, the teacher does not know if the September 26–28, 2017, 10 pages. DOI: 10.1145/3129416.3129427 learner guessed the answer or applied the divisibility rule correctly. Permission to make digital or hard copies of all or part of this work for personal or This study makes use of eye-tracking technology to observe classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation learners’ gaze behaviour while they are applying the divisibility on the first page. Copyrights for components of this work owned by others than ACM rules. The research question that will be addressed is if it is possible must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, that learners’ gaze behaviour can indicate whether they applied the to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. divisibility rules correctly when they correctly indicated if a SAICSIT '17, September 26–28, 2017, Thaba Nchu, South Africa dividend is divisible by a specific single digit divisor. © 2017 ACM. 978-1-4503-5250-5/17/09. . . $15.00 https://doi.org/10.1145/3129416.3129427 SAICSIT 2017, September 2017, Thaba 'Nchu, South Africa P.H. Potgieter & P.J. Blignaut

The rationale of the study will be provided in the following 2.4 True/false questions section. Thereafter, previous attempts where eye-tracking was The probability of guessed answers for true/false questions is high used to analyse gaze behaviour while participants solve [21, 22]. The credibility of true/false questions can be improved by mathematical problems, will be discussed. This will be followed by asking learners to provide reasons for their answers [8]. However, the experimental details. The results will refer to the the effect of when learners are expected to provide a reason for an answer, they grade, revision and divisor on the percentage of fixation time per may have difficulty in expressing how they reached a specific digit as well as the gaze behaviour of learners who provide incorrect answer [23]. Each class has its “Lucky Larry”, who manages to get answers. The minimum attention required per digit per divisor will the correct answer from incorrect reasoning [24]. Learners want to be investigated and some conclusions will be made. provide satisfactory answers to questions and they will often act as if they know the correct strategy – although there is no evidence 2 RATIONALE OF THE STUDY that they are aware of it [25]. In other words, learners may seem Performing can be time consuming and complex. If, confident about their reasoning even though the reasoning is however, the quotients or remainders are not required, divisibility incorrect [26, 27]. The use of certain strategies indicates the rules can be used to determine if one is divisible by another understanding of mathematical concepts and learners should have [9]. “a divides b if and only if a is a factor of b. When a divides b, the ability to recognise where and when specific strategies could be we can also say that a is a divisor of b, a is a factor of b, b is a used to simplify processing of a problem [28]. Teachers cannot multiple of a, and b is divisible by a” [10]. Divisibility rules present identify the incorrect application of a divisibility rule if the correct a short way of determining whether a divides b without actually answer and motivation were provided. performing the division. 3 GAZE BEHAVIOUR 2.1 Basic background Eye movements reveal detailed information on the procedure of Rules can be used for problem solving, but it must be applied with how a learner approaches a problem and how the learner reaches understanding [11]. The concepts of division and divisibility are the solution to the problem [29]. Since most eye-tracking systems important factors that help learners move from Arithmetic to make use of infrared illumination that is invisible to the eye and Algebra [12]. does not distract or annoy the user [30], eye-tracking can be used Knowledge of the divisibility rules can make life easier for as a non-intrusive technology to investigate learners’ gaze learners in various application areas. Some of these areas include behaviour [31] while they are doing Mathematics. finding factors of a dividend, determining if a dividend is a prime number, calculating factor pairs of a dividend, simplifying fractions 3.1 Fixations and calculating the lowest common multiple and least common A fixation is defined as the maintaining of the visual gaze on a single denominator of two dividends. location, and a saccade is a quick motion of the eye from one 2.2 Teaching fixation to another [32]. The duration of a fixation is at least 100 to 150 milliseconds and a fixation represents the gaze on an attention The teachers’ role is to transfer knowledge and mathematical rules area [30]. As such, eye-tracking can be used to inspect participants’ to learners [13] and provide guidance that encourage learners to distribution of visual attention while they are solving problems express themselves and improve their reasoning skills [14-16]. [29]. Eye-tracking can thus also reveal which areas participants are Teachers can determine if learners grasped certain concepts and are not inspecting [33]. The correct patterns of gaze behaviour could be in a position to decide when to intervene if learners struggle with associated with learners who possess solid factual knowledge of a specific concepts [17]. Learners’ understandings of mathematical problem [34]. Analysis of gaze behaviour can be used to identify the concepts improve when they recognised their misunderstandings academic potential and cognitive activities of learners [31, 35-37]. or mistakes, or when the teacher point out their mistakes or There is also huge potential in investigating how complex misunderstandings [18]. numerical tasks are performed and how the strategies of well- performing learners differ from those of poor-performing learners 2.3 Problem solving [38]. Participants who know how to perform a task, spend more There are different types of knowledge that learners could use to time on relevant areas than on irrelevant areas [39] and shift their solve a mathematical problem. The most common types of gaze and attention after practice to more appropriate areas of knowledge are conceptual and procedural knowledge. Procedural interest [40]. knowledge can be classified as a series of steps, or actions, done to accomplish a goal [19]. When learners use rules to solve 3.2 Eye-tracking complements verbal responses mathematical problems, it can be classified as procedural Gaze behaviour cannot be obtained from verbal responses [41] and knowledge [20]. The action to determine divisibility without the verbal explanation of reasoning can on its own not be used to actually doing it, is a type of procedural activity and procedural derive which strategies were used [33]. Verbal responses, together understanding [9]. with eye movement data, can be used to examine cognitive processes [41]. Learners’ eye movements can potentially provide

Using Eye-Tracking to Assess the Application of Divisibility Rules SAICSIT 2017, September 2017, Thaba 'Nchu, South Africa more information than traditional written assessments and also control group and the same learners participated before and after reveal the strategies that learners use when dealing with the divisibility rules were explained or revised. mathematical problem solving [39]. In addition, learners’ gaze behaviour might also be used to investigate if there were implicit 4.1 Assessments forms of understanding that accompany incorrect verbal responses Seventy-eight learners from Grade 4 to Grade 7 participated in two [42]. During the capturing of eye movements, participants could be assessments with eye-tracking. Each assessment focused on the asked additional questions in order to supplement the data [43]. ability of the learners to apply the divisibility rules on divisors 2 to 9, excluding 7. Divisor seven (7) was excluded because the rule for 3.3 Peripheral vision divisibility is more complicated than the rules for the other divisors A specific limitation of eye-tracking, known as the dissociation in the range. problem, is that participants may fixate on a certain area while they The first assessment was an unprepared test where learners had are paying attention to another area [43]. Visual attention may be to verbalise their answers and also provide reasons while looking at overt or covert [44]. Participants may use peripheral vision to a series of five-digit dividends one after the other on a computer inspect areas of interests that they do not directly fixate on [23]. screen. Revision was done on the divisibility rules a month after the Information within an AOI, which were observed by peripheral first assessment. The second assessment was done a week after the vision or direct fixations, can be absorbed by the short-term revision lesson, in the same way as the first assessment. Only memory if it stays there long enough to be recalled [45]. learners who participated in both assessments were used for the There are limitations when eye-tracking is used to investigate analysis. learners’ reasoning while they are completing mathematical Fourteen questions, two per divisor, were asked in random order assessments, because one can only determine where they are in each of the two assessments. The same set of questions was used looking and not what they are thinking at any specific moment [43]. for both assessments. For each divisor there was a question where Although gaze behaviour is indicative of cognitive processes, it will the dividend was divisible by the divisor and a question where the not be fully known what the brain absorbs while the participant is dividend was not divisible by the divisor. fixating on an object [43]. Eye-tracking is not an alternative for any A learner should be able to answer three true/false questions or other method of assessment, but it could be used in combination three short-answer questions per minute [57] and therefore each with other methods, such as verbal responses, to analyse thought dividend was displayed for 20 seconds. The researcher initiated the processes [17, 43, 46]. move to the next stimulus as soon as the learner provided an answer or after 20 seconds have expired. The researcher also recorded the 3.4 Attempts to identify gaze behaviour in learners’ verbal responses. mathematical problem solving Although many eye-tracking studies have already been performed 4.2 Compilation of Dividends on graphs, formulas, geometry, etc. [25, 32, 47-52], the discussions All the dividends that were presented to learners were five-digit in this section will focus on the use of eye-tracking in applicable numbers as Grade 4 learners are supposed to identify a five-digit mathematical areas relating to the divisibility rules. number [58]. Godau [53] and Godau et al. [28] conducted a study to determine During the pilot phase of the study, it was observed that learners the ability of Grade 1 to Grade 3 learners to spontaneously notice inspected only the last digit or the number formed by the last two commutativity (a+b+c = a+c+b) and it was concluded that learners digits if they did not know the divisibility rule. Therefore, most of used search processes to identify a suitable strategy to use and that the dividends were compiled such that when learners inspected better understanding of concepts led to an improvement in the only the last digit or the number formed by the last two digits, the strategy used and vice versa. There was no evidence that the order answer would probably be incorrect. and number of fixations that participants moved their gaze had an A calculator was not allowed and the choice of digits in the effect on whether or not the correct strategy was used [54]. Previous dividends was such that learners could do mental calculations. All research studies found that the magnitude of numbers determine dividends were chosen such that the sum of all digits varied from fixation duration [44]. 18 to 20 (for example, 6+3+1+7+2=19). Cimen [55] performed a case study on division theory where (i) mathematical instruction was offered to elucidate the relationship 4.3 Stimuli between a divisor, dividend, quotient and remainder; and (ii) where The stimuli were presented as slides in landscape format and participants were assessed on the learning material and it was found optimised for the laptop display with a resolution of 1600900 pixels that participants built connections between related concepts of (36.09°×20.77°). “Digit 5” refers to the leftmost digit, while “Digit 1” divisibility. refers to the rightmost digit. The A and B “digits” were placed at the sides of the dividend to minimise the positional advantage that the 4 EXPERIMENTAL DETAILS first and last digits would enjoy (Figure 1). The general pre-post experimental design requires the same group of individuals to be measured prior to and after treatment [56]. This design was chosen for this study because there was no

SAICSIT 2017, September 2017, Thaba 'Nchu, South Africa P.H. Potgieter & P.J. Blignaut

quality of the eye-tracking data was such that there were no fixations on the areas of interest, were discarded. Two questions were presented to learners for every divisor: one question where the dividend was divisible by the divisor and one question where it was not divisible. Using a within-subjects one- way analysis of variance, it was found that there is no significant (α = .05) difference in the percentage of learners who provided the correct answer and reason (AR) if a dividend is divisible by a Figure 1: Areas of interest divisor or not and therefore the influence of divisibility was discarded for further analysis. The dividends, with spaces between the digits, were displayed in an Arial typeface with font size 72 (0.88°1.32°). The dividend and 5.1 Effect of Grade on Percentage of Fixation all characters were spread out evenly across the display [59]. The Time distance between the digits of the dividend was 3.97. The results of a within-subjects one-way analysis of variance Each digit was included in an area of interest (AOI) of 160160 (ANOVA) for the effect of the learner's grade on the percentage of pixels (3.263.26) (Figure 1). AOIs of 80160 pixels (1.633.26) fixation time (%) while controlling for the effect of revision, divisor were inserted between the digits to capture fixations between digits. and digit, were investigated. With the exception of four of the 70 AOIs of 80160 pixels were also placed at the ends of the dividend. cases (7 divisors, 5 digits, before and after revision), it was found The fixation time on the AOIs were used to calculate the percentage that the effect of grade on the percentage of time that learners fixate of total fixation time on the entire dividend for each one of the five on the respective digits, was not significant (α = .05). digits. Fixations on the AOIs between digits contributed 50% to each of the digits on either side. 5.2 Effect of Revision on Percentage of Fixation The instruction for each question was initially displayed without Time the dividend, for example “Is the following number divisible by 2?” 5.2.1 Learners who performed better after revision. Learners who Once the learner had time to read the instruction, the dividend was provided an incorrect reason for a question in the first assessment, displayed. The instruction was still visible in case the learner forgot irrespective of what their answers were (AR× or A×R×), did not which divisor to use, but it was displayed in grey to minimise know the relevant divisibility rule. If the same learners provided the distraction from the dividend. As soon as a learner verbalised correct answer and reason for the same question after revision his/her answer (“yes” or “no”), the dividend was removed and the (AR), they probably benefited from the revision session and learner was requested to provide a reason for his/her answer. would now be able to apply the divisibility rule. Table 2 shows the results of a within-subjects repeated-measures 4.4 Research instruments ANOVA for the effect of revision on the percentage of fixation time A 60 Hz Tobii X2-60 eye-tracker was used to capture the gaze while controlling for divisor and digit for learners who benefited behaviour of learners. Tobii Studio (version 12) [60] was used to from the revision (learners in AR× or A×R× before revision and obtain the percentage of total fixation time on each AOI around the AR after revision). N indicates the number of responses (there digits of the dividend for all the eye-tracking recordings. These were two questions per divisor, thus two responses per learner) in percentages, together with learners’ responses (answer and reason), the respective groups. were entered into an MS Excel spreadsheet to prepare the data for No significant difference (α = .05) between the percentage statistical analysis. A nine point calibration was done prior to the fixation time on the A and B “digits” before and after revision was assessment to ensure good quality of the eye gaze recordings. found for anyone of the divisors. There were also no significant differences (α = .05) for divisor 2. This could be due to the trend 5 EFFECT OF GRADE, REVISION AND DIVISOR (Section 6.2) that if learners did not know the reason, they fixated mainly on the last two digits of the dividend which correlates with ON FIXATION TIME PER DIGIT the divisibility rule of divisor 2. Although the same argument The effect of learner's grade, revision and divisor on the percentage applies to divisor 5, a significant p value (α = .05) was found on of fixation time per digit are analysed below for learners who digit 1 for divisor 5. Significant p values (α = .05) were found for all provided the correct answer and reason (AR). These learners the digits for divisors 3, 6 and 9 except for divisor 3, digit 3. probably applied the divisibility rules correctly and fixated on the Significant p values (α = .05) were found for some of the digits for required digits to conclude their answers. Learners who provided divisors 4 and 8. Therefore, it can be surmised that revision has a an incorrect reason (AR×) probably did not know the relevant significant impact on the percentage of fixation time for learners divisibility rule and their fixations on the digits would not be who benefited from the revision. reliable. Learners who provided a correct motivation but an 5.2.2 Learners who provided the correct answer and reason before incorrect answer (A×R) probably made mental calculation errors and after revision. If the same learners provided the correct answer or applied the divisibility rule incorrectly and therefore their and reason in Assessment 1 and again in Assessment 2, it probably fixations on the digits could be misleading. Recordings where the means that they knew the divisibility rules before revision and that

Using Eye-Tracking to Assess the Application of Divisibility Rules SAICSIT 2017, September 2017, Thaba 'Nchu, South Africa

Table 2: The effect of revision on the percentage of 5.3 Effect of Divisor on Percentage of Fixation fixation time on a digit for learners who Time benefited from revision. N indicates the number of responses in the respective groups (AR× or 5.3.1 Answer and reason correct (AR). When learners A×R× before revision and AR after revision) provided a correct answer and reason, irrespective of whether it was before or after revision, one can infer that the learners knew

Percentage of total the divisibility rule and also how to apply it. A series of one-way fixation time analyses of variance (ANOVA) showed a significant (α = .001) Before After

Divisor Digit effect of divisor on the percentage of fixation time on each one of N revision revision Effect p A&B 0.25 - the five digits for both assessments for learners who provided the 5 19.21 - correct answer and reason. This result is no surprise as it is 4 10.18 0.79 F(1, 6)=3.10 .129 2 7 expected, for example, that learners have to focus on all digits for 3 7.88 1.35 F(1, 6)=3.14 .127 divisibility by 3 while they only need to inspect the last digit to 2 23.16 35.32 F(1, 6)=1.08 .339 1 39.33 62.54 F(1, 6)=2.56 .161 determine if a dividend is divisible by 2 or by 5. A&B 0.23 0.07 F(1, 69)=2.59 .112 5.3.2 More answer/reason combinations. The previous analysis 5 1.99 13.28 F(1, 69)=48.01 .000 was performed for learners in AR. When learners provided an 4 6.49 27.51 F(1, 69)=119.24 .000 3 70 incorrect reason (R×), one can infer that the learners did not know 3 18.35 22.18 F(1, 69)=3.71 .058 2 44.17 24.17 F(1, 69)=68.25 .000 the divisibility rule and guessed the answer. When learners 1 28.78 12.73 F(1, 69)=30.68 .000 provided an incorrect answer but a correct reason (A×R), it could A&B 0.34 0.05 F(1, 31)=1.15 .292 mean that the learners knew the divisibility rule but probably 5 2.39 3.37 F(1, 31)=.21 .653 made mental calculation errors or applied the rule incorrectly. 4 3.28 2.33 F(1, 31)=.42 .520 4 32 3 7.98 3.65 F(1, 31)=4.34 .046 This was confirmed when learners verbalised their calculations 2 49.50 52.83 F(1, 31)=.33 .569 aloud. Examples of such verbalisations were: "75133 are divisible 1 36.52 37.76 F(1, 31)=.05 .819 by 3 since 7+5+1+3+3 = 18” and “3 is a factor of 19”. A&B 0.04 - Table 3 shows the average percentage of fixation time of 5 6.97 - 4 14.68 - learners in the different answer/reason combinations per digit and 5 3 3 14.80 - divisor. The maximum possible value of N is 312 (78 learners took 2 40.63 8.88 F(1, 2)=2.41 .261 part in 2 assessments × 2 questions per divisor). As mentioned 1 22.89 91.12 F(1, 2)=163.08 .006 before, recordings where the quality of the eye-tracking data was A&B 0.19 0.13 F(1, 53)=.37 .548 5 2.77 10.59 F(1, 53)=38.93 .000 such that there were no fixations on the areas of interest were 4 3.33 16.08 F(1, 53)=42.83 .000 discarded. Table 3 shows that if a learner provided an incorrect 6 54 3 13.87 20.13 F(1, 53)=5.74 .020 answer and reason (A×R×), the most attention was on digit 1 and 2 48.11 31.66 F(1, 53)=26.10 .000 1 31.74 21.42 F(1, 53)=7.79 .007 digit 2 – irrespective of the divisor. Learners who provided the A&B 0.15 0.06 F(1, 31)=.56 .459 correct answer but with a wrong explanation of how they arrived 5 6.32 0.78 F(1, 31)=6.45 .016 at their answers (AR×) also focused mainly on digit 1 and digit 2. 4 8.04 2.81 F(1, 31)=5.29 .028 8 32 3 19.29 29.91 F(1, 31)=7.10 .012 2 45.40 51.30 F(1, 31)=1.46 .236 6 GAZE BEHAVIOUR OF LEARNERS WHO 1 20.79 15.14 F(1, 31)=2.79 .105 PROVIDE INCORRECT ANSWERS A&B 0.13 0.20 F(1, 77)=.38 .538 5 3.44 10.95 F(1, 77)=37.28 .000 4 6.78 15.98 F(1, 77)=33.21 .000 6.1 The Effect of Correctness of Answer on the 9 78 3 16.51 22.79 F(1, 77)=10.11 .002 Percentage of Fixation Time per Digit 2 44.06 29.92 F(1, 77)=34.81 .000 (A×R vs AR) 1 29.07 20.17 F(1, 77)=10.16 .002 Learners who apply a divisibility rule correctly (R) should revision was actually unnecessary. The results of a within-subjects focus on the correct digits, irrespective of whether their answers repeated-measures ANOVA for the effect of revision on the are correct or not. A series of one-way ANOVAs (excluding percentage of fixation time on the respective digits for learners in divisor 2 and controlling for divisor and digit) confirmed that, AR at both occasions were investigated. As expected, the results with the exception of divisor 6, correctness of the answer had no show that learners who knew the divisibility rules before revision significant (α = .01) effect on the percentage of fixation time per spent nearly the same percentage of fixation time on the different digit. Divisor 2 was not included because no participants applied digits before and after revision. Significant p values (α = .01) were the rule correctly along with an incorrect answer. only found for digits 1, 4 and 5 for divisor 2. Therefore, it seems that revision does not have an influence on the percentage of time spent on a digit if learners knew the divisibility rule before revision.

SAICSIT 2017, September 2017, Thaba 'Nchu, South Africa P.H. Potgieter & P.J. Blignaut

Table 3: Answer/reason combinations with percentage fixation time per digit. (Key: A=Answer; R=Reason; =Incorrect; =Correct; N=number of responses)

Divisor 2 3 4 5 6 8 9 Combination Digit N % N % N % N % N % N % N % A & B 0.32 0.09 0.15 0.28 0.26 0.10 0.17 5 2.65 11.59 1.90 2.65 10.47 0.51 10.28 4 2.95 29.45 2.45 2.67 16.60 3.02 15.83 AR 294 105 111 292 84 61 105 3 5.91 22.29 7.59 4.70 18.56 32.22 23.36 2 33.95 24.41 56.94 39.32 31.35 52.03 30.11 1 54.23 12.17 30.98 50.38 22.76 12.11 20.25 A & B 0 0.28 0.59 0 0.14 0.16 0.37 5 18.57 2.25 3.72 0 2.62 3.96 2.94 4 2.94 5.32 5.08 1.20 5.11 6.66 5.05 AR 11 166 137 9 142 172 164 3 4.87 11.66 8.19 21.08 11.47 15.97 13.08 2 20.59 46.73 43.13 44.68 46.84 43.92 40.28 1 53.02 33.78 39.29 33.04 33.83 29.33 38.29 A & B 0 0.49 0.28 0 0.60 0.10 0.09 5 12.66 3.76 4.32 3.53 5.77 4.20 3.87 4 10.72 10.00 2.75 6.78 8.44 6.89 8.95 AR 5 21 39 6 47 45 27 3 5.19 24.97 9.42 8.93 11.20 13.80 16.70 2 37.17 37.06 42.57 31.30 41.60 45.85 50.22 1 34.27 23.72 40.66 49.49 32.41 29.16 20.16 A & B - 0 0.25 0 0.33 0.15 0 5 - 9.73 2.74 0 5.45 0.42 7.35 4 - 38.98 4.46 0 7.17 3.25 13.16 AR 0 14 21 1 33 26 10 3 - 22.49 7.04 2.18 11.69 25.87 20.10 2 - 20.85 51.82 34.15 34.87 50.05 39.79 1 - 7.94 33.69 63.62 40.49 20.25 19.60

Special case for divisor 6. In the first assessment, some learners stated the divisibility rule for divisor 6 correctly, but when they were prompted for more details, it became evident that they only inspected the last one or two digits of the dividend. The dividends for divisor 6 were compiled such that inspection of only the last or last two digits would result in an incorrect answer. Table 4 indicates the percentage of fixation time on the digits for divisor 6 where learners provided the correct divisibility rule before and after revision, but indicated an incorrect answer before revision (A×R) and a correct answer after revision (AR). The results of Figure 2: Percentage of total fixation time per divisor and a within-subjects repeated-measures ANOVA for the effect of digit for learners in School A in A×R× (Both assessments) revision on percentage of fixation time are also shown. Significant p values (α = .05) were found for all the digits except therefore, infer that the percentage of fixation time on digits can for digit 2. Therefore, it can be inferred that some learners who indicate if a learner applied the rule correctly. knew the divisibility rule for divisor 6, applied the rule incorrectly (did not actually focus on all digits) before revision. One can, 6.2 Percentage of Fixation Time per digit for Learners in A×R× Table 4: The effect of revision on the percentage of fixation time per digit for divisor 6 for learners A trend was observed that if a learner provided an incorrect answer in A×R before revision and AR after and did not know the divisibility rule, the highest percentage of revision. (N=number of responses) fixation time was mainly on the last two digits (digit 1 and digit 2). Although they indicated verbally that they only inspected the last Percentage of fixation time digit of the dividend, it was revealed through eye-tracking that Assessment 1 Assessment 2 there was also a high percentage of fixation time on the second-to-   AR

Divisor Digit N A R p last digit. A & B 0.43 0.15 .569 Information gained from participants is not very objective and 5 2.33 9.81 .033 4 3.68 18.00 .018 eye-tracking can be a useful complementary source of information 6 11 3 6.04 15.71 .004 to identify the strategies that participants use when solving a 2 32.51 26.39 .371 problem [24]. Therefore, when teachers compile dividends to be 1 55.01 29.93 .004 used in the assessment of divisibility rules, they have to keep in

Using Eye-Tracking to Assess the Application of Divisibility Rules SAICSIT 2017, September 2017, Thaba 'Nchu, South Africa mind that learners who do not know the divisibility rules, mainly Table 5: Percentage of fixation time per divisor and digit inspect the last two digits of the dividend. for learners in AR. (Learners should inspect The average percentage of fixation time per digit for learners the underlined digits according to the divisibility who provided the incorrect answer and also the incorrect rule) divisibility rule (A×R×) is shown in Table 3. Figure 2 provides a Percentage fixation time visualisation of the results for learners in A×R×. A series of one- Divisor % Lower limit of 95% way analyses of variance (ANOVA) showed a significant effect of N Digit confidence interval the digit position on the percentage of fixation time for each divisor. A&B 0.32 0.17 This means that learners were deliberately focusing on the last two 5 2.65 1.81 4 2.95 2.13 digits irrespective of what the divisor was. 2 294 3 5.91 4.82 2 33.95 31.16 7 MINIMUM ATTENTION REQUIRED PER 1 54.23 51.04 DIVISOR A&B 0.09 0.02 5 11.59 9.31 Table 5 shows the average percentage of fixation time that learners 4 29.45 26.19 3 105 who provided the correct answer and reason spent per digit. The 3 22.29 19.88 last column in the table indicates the lower limit of the 95% 2 24.41 21.82 confidence interval, i.e. the percentage of which we can be 95% sure 1 12.17 10.19 A&B 0.15 0.02   that learners in A R spent at least this much time on the 5 1.90 0.82 respective digits. Using these values and the trends that were 4 2.45 1.40 4 111 discovered above, the minimum required attention levels per digit 3 7.59 5.13 can be determined for each divisor. Based on these levels, a teacher 2 56.94 53.20 will be able to tell if the respective divisibility rule was applied 1 30.98 27.22 correctly. A&B 0.28 0.11 5 2.65 1.74 4 2.67 1.86 5 292 7.1 Minimum Attention Required per Divisor 3 4.70 3.46 7.1.1 Divisors 2 and 5. A dividend is divisible by 2 if the last digit is 2 39.32 36.31 even, and divisible by 5 if the last digit is zero (0) or five (5). Since 1 50.38 47.08 both the rules for divisors 2 and 5 expect the learners to focus on A&B 0.26 0.03 5 10.47 8.71 the last digit only, it could be argued that the same criteria should 4 16.60 14.04 6 84 be applied for these two divisors. Table 5 indicates that the average 3 18.56 16.42 percentage of fixation time that learners spent on the last digit for 2 31.35 28.27 divisor 2 and 5 was 54.23% and 50.38% respectively and that we can 1 22.76 19.00 be 95% sure that the percentage of fixation time will be higher than A&B 0.10 0.04 51.04% and 47.08% respectively. 5 0.51 0.14 4 3.02 1.31 The minimum percentage of fixation time of 47% will be 8 61 acceptable for divisors 2 and 5. Therefore, one can surmise that for 3 32.22 27.37 2 52.03 47.27 divisibility by divisors 2 and 5 learners had to spend 47% of their 1 12.11 8.70 time on the last digit to be 95% sure that the divisibility rule was A&B 0.17 0.04 applied correctly. 5 10.28 8.38 7.1.2 Divisor 4. A dividend is divisible by 4 if the number formed 4 15.83 13.96 9 105 by the last two digits is divisible by 4. Table 5 indicates that the 3 23.36 20.97 average percentage of fixation time that learners in AR spent on 2 30.11 27.55 the last two digits for divisor 4 was 87.92% and that we can be 95% 1 20.25 17.61 sure that the total fixation time on the last two digits will be higher than 80.42%. Therefore, one can surmise that for divisibility by 4, where it can be seen in Table 5 that the percentage of fixations on learners had to fixate on both digit 1 and digit 2, and they had to the last digit was much lower than on digits 2 and 3. Therefore, one spend 80% of the time on the last two digits to be reasonably sure can surmise that for divisibility by divisor 8, learners had to spend that the learner applied the divisibility rule correct. 80% of the time on the combined last three digits for the researcher 7.1.3 Divisor 8. A dividend is divisible by 8 if the number formed to be 95% sure that the learner applied the divisibility rule correctly, by the last three digits is divisible by 8. Table 5 indicates that the and the learners also had to fixate on digit 2 and digit 3. average percentage of fixation time that learners in AR spent on 7.1.4 Divisor 3, 6 and 9. A dividend is divisible by 3 if the sum of the last three digits for divisor 8 was 96.36% and that we can be 95% the digits is divisible by 3 which means that a learner should inspect sure that the total fixation time on the last three digits will be higher all the digits. The percentage of fixation time on the digits vary than 83.34%. All the learners in AR fixated on digit 2 and digit 3, between the two questions that were asked, but all digits enjoyed at but 9% of these learners did not fixate on digit 1. This agrees with least 10% of the attention. The smallest percentage of fixation time

SAICSIT 2017, September 2017, Thaba 'Nchu, South Africa P.H. Potgieter & P.J. Blignaut was on digits 1 and 5. It is understandable that there will be a low guessed the correct answer. False rejects would occur when the percentage of fixation time on the first digit that learners inspect, learners’ gazes (G) show too little attention on the respective digits regardless of whether they start from the left or right, because they but both the answer (A) and reason (R) is correct (example Figure only have to remember the first digit. Thereafter, learners have to 3g). False accepts can occur when the relevant digits enjoy an accumulate the values of the digits – thus adding to the mental acceptable amount of attention but the answer and/or the effort with consequential longer fixations. Although every digit motivation is incorrect. The overall success rate where learners enjoyed at least 10% attention, it will not be justified to require a provided the correct answer and where the attention corresponded minimum percentage of fixation time per digit. If a digit is zero (0) with the minimum required attention levels (or the lack thereof in or one (1), learners will quickly fixate on it and move on. It is, AR), was 85.74%. In 8.6% of the cases, incorrect applications of however, important that learners fixate on all the digits. the divisibility rules were predicted to be correct and in 5.7% of the A dividend is divisible by 6 if the dividend is an even number cases, correct applications were predicted to be incorrect. The false and the sum of the digits is divisible by 3, which means that the accepts and false rejects for incorrect answers were not calculated, learner should inspect all the digits as for divisibility by 3. Because because the learners were not going to receive any marks for of the general trend that learners who do not know the divisibility incorrect answers. rule fixate mainly on the last two digits, it is important that the dividend is even when testing divisibility by 6. 8 SUMMARY AND CONCLUSIONS A dividend is divisible by 9 if the sum of the digits is divisible by 9. This means that learners had to inspect all digits. The same When teachers assess learners by using true/false (“yes” or “no”) arguments that hold for divisor 3 also apply to divisor 9. questions, they do not have access to the learners’ reasoning. This Although learners should inspect all the digits for divisors 3, 6 study investigated whether eye-tracking can assist teachers in (if it is even) and 9, it was found that some learners in AR did discovering learners’ reasoning while they are thinking about their not do so. Some learners fixated on four of the five digits only. It answers. could be argued that the fixations were too short to be registered by Learners were presented with a series of five-digit dividends on the eye-tracker (as in cases where easy digits, such as 0 or 1, were a computer screen while they had to determine whether the not fixated on), or it could be that digits are perceived in learners’ dividends were divisible by divisors 2, 3, 4, 5, 6, 8 and 9. peripheral vision without explicit fixations. It could, therefore, be After revising the divisibility rules, the assessment was repeated argued that in order to determine divisibility by 3, 6 (if the dividend for the same learners and the effects of learners' grade, revision, is even) or 9, learners have to fixate on at least four of the five digits. divisor and correctness of answer on gaze behaviour were determined through a series of within-subject analyses of variance. 7.2 Validation of the Minimum Required It was found that learners' grade had no significant effect on the Attention Levels percentage of time that learners fixate on the respective digits. Revision proved to be a significant indicator of gaze behaviour for To validate the required minimum attention levels that were set, all learners who improved their performance after revision but not for the eye-tracking recordings with the learners’ answers and reasons learners who knew the divisibility rules before revision. The divisor were used to compare these levels with the reasons that learners affected gaze behaviour significantly as it is not always necessary provided. Although recordings were discarded in the analysis of to inspect all digits. Learners who could not explain the divisibility data where there were no fixations on the areas of interest, it was rule, spent most attention on the last two digits – irrespective of used during the validation because it could happen in practice that whether the answer was correct (probably guessed) or not. With the there are no fixations on the AOIs. Table 6 shows the percentage of exception of divisor 6, no difference in gaze behaviour was detected responses in each of the possible answer/reason/gaze combinations for learners who explained the rule correctly – again irrespective of combined for all divisors per grade for the two assessments. whether the answer was correct or incorrect (probably due to a Figure 3 provides a visualisation of the results in Table 6. calculation error). One can, therefore, infer that the percentage of If a learner provides an incorrect answer, a teacher must fixation time on digits can indicate if a learner applied the rule determine the reason, because it is possible that the learner (i) does correctly. not know the divisibility rule (Figure 3a), (ii) knows the divisibility For each divisor, a minimum percentage of fixation time per digit rule but applied it incorrectly (Figure 3c), or (iii) knows the were determined that could be used by teachers to tell if the divisibility rule but made calculation errors (Figure 3d). For divisors respective divisibility rule was applied correctly. It was found that 3, 6 and 9, the percentage of fixations could readily indicate which for the learners who provided the correct answer in this study, these one of these categories applies. For the other divisors, it is less values were accurate in about 85% of the cases to indicate whether obvious since the trend that learners who do not know the the learners also applied divisibility rule correctly. divisibility rule focus on the last two digits, overlaps with the In summary, it is concluded that when a teacher is in possession correct gaze behaviour. Figure 3b illustrates this trend, because of the learner’s answer, reason and fixation data, the teacher is in a there were a percentage of responses in A×R× with acceptable gaze position to identify if the learner (i) guessed the answer, behaviour. It is therefore important that the teacher prompts the (ii) correctly applied the divisibility rule, (iii) correctly applied the learners for a motivation if their answers are incorrect. divisibility rule but made mental calculation errors, or (iv) In cases where learners does not know the divisibility rule but incorrectly applied the divisibility rule. Eye-tracking can assist provided the correct answer (Figure 3e and 3f) they probably

Using Eye-Tracking to Assess the Application of Divisibility Rules SAICSIT 2017, September 2017, Thaba 'Nchu, South Africa

Table 6: Percentage of responses in each of the possible answer/reason/gaze combinations combined for all divisors per grade for the two assessments. (Key: A=Answer; R=Reason; G=Gaze behaviour; =Incorrect/Inadequate; =Correct/Adequate)

Percentage Assessment 1 Assessment 2 Grade                                         A R G A R G A R G A R G A R G A R G A R G A R G A R G A R G A R G A R G A R G A R G A R G A R G 4 41.8 20.0 0.0 0.4 5.7 5.0 1.4 25.7 26.4 14.3 0.4 3.9 5.0 7.1 2.5 40.4 5 27.0 25.4 2.8 2.8 4.4 4.8 3.6 29.4 12.7 17.9 0.8 2.8 2.0 6.3 3.2 54.4 6 30.7 23.9 2.5 2.5 2.5 5.0 3.6 29.3 8.9 12.9 0.4 6.8 2.9 0.7 3.2 64.3 7 18.6 17.9 0.7 2.9 5.7 6.4 4.3 43.6 3.9 5.4 1.1 5.4 2.1 3.9 4.3 73.9

a b c d

e f g h Figure 3: Percentage of responses per grade, answer, reason and gaze for all the divisors with minimum gaze requirements teachers in identifying whether learners applied the divisibility [8] Singh, U.G. 2014. The development of a framework for evaluating e-assessment systems (Doctoral dissertation. University of South Africa). Retrieved from rules correctly when their answers were correct. However, due to http://uir.unisa.ac.za/handle/10500/14619 (Accessed on 16 April 2016) the relatively high probability of error, eye-tracking cannot be used [9] Zazkis, R., & Campbell, S. 1996. Divisibility and multiplicative structure of exclusively to determine if learners applied the divisibility rules natural numbers: Preservice teachers' understanding. Journal for Research in Mathematics Education 27(5), 540-563. correctly or not. [10] Musser, G.L., Burger, W.F. & Peterson, B.E. 2011. Mathematics for elementary teachers. Ninth Edition. John Wiley & Sons (Asia) Pty Ltd. Asia. [11] Ploger, D., & Rooney, M. 2005. Teaching fractions: Rules and reason. Teaching REFERENCES Children Mathematics, 12(1), 12-17. [12] Cimen, O.A., & Campbell, S.R. (2013). Qualitative learner profiling: Basic [1] Department of Basic Education. 2013. Report on the Annual National Assessment concepts of elementary number theory. In 2013 Canadian Society for the Study of 2013. Pretoria: Department of Basic Education. of Education. [2] Department of Basic Education. 2014. Report on the Annual National Assessment [13] Nisbet, S., & Warren, E. 2000. Primary school teachers’ beliefs relating to of 2014. Pretoria: Department of Basic Education. mathematics, teaching and assessing mathematics and factors that influence [3] Department of Basic Education. 2013c. Annual National Assessment: 2013 these beliefs. Mathematics Teacher Education and Development, 2(34-47). Diagnostic Report and 2014 Framework for improvement. Pretoria: Department [14] Baig, S., & Halai, A. 2006. Learning mathematical rules with reasoning. Eurasia of Basic Education. Journal of Mathematics, Science and Technology Education, 2(2), 15-39. [4] Department of Basic Education. 2014a. The Annual National Assessment of 2014: [15] Maher, C., & Weber, K. 2009. Precision in the teaching, learning, and Diagnostic Report Intermediate and Senior phases Mathematics. Pretoria: communication of elementary school Mathematics: A reply to Wilson's. AASA Department of Basic Education. Journal of Scholarship & Practice, 6(1), 50-54. [5] Department of Basic Education. 2015. Grade 5 - Mathematics in English, Book [16] Witzel, B.S., & Riccomini, P.J. 2007. Optimizing math curriculum to meet the 1, Terms 1 & 2. Fifth edition. Pretoria. South Africa. learning needs of students. Preventing School Failure: Alternative Education for [6] Department of Basic Education. 2015. Grade 6 - Mathematics in English, Book Children and Youth, 52(1), 13-18. 1, Terms 1 & 2. Fifth edition. Pretoria. South Africa. [17] Van den Bogert, N., Van Bruggen, J., Kostons, D., & Jochems, W. 2014. First [7] Department of Basic Education. 2015. Grade 7 - Mathematics in English, Book steps into understanding teachers' visual perception of classroom events. 1, Terms 1 & 2. Fifth edition. Pretoria. South Africa. Teaching and Teacher Education, 37, 208-216.

SAICSIT 2017, September 2017, Thaba 'Nchu, South Africa P.H. Potgieter & P.J. Blignaut

[18] Tan, S.F., & Lim, C.S. 2010. Effective Mathematics lesson from the lenses of [42] Heine, A., Thaler, V., Tamm, S., Hawelka, S., Schneider, M., Torbeyns, J., ... & primary pupils: Preliminary analysis. Procedia-Social and Behavioral Sciences, 8, Jacobs, A.M. 2010. What the eyes already ‘know’: using eye movement 242-247. measurement to tap into children's implicit numerical magnitude [19] Rittle-Johnson, B., Schneider, M., & Star, J.R. 2015. Not a one-way street: representations. Infant and Child Development, 19(2), 175-186. Bidirectional relations between procedural and conceptual knowledge of [43] Bolden, D., Barmby, P., Raine, S., & Gardner, M. 2015. How young children view mathematics. Educational Psychology Review, 27(4), 587-597. mathematical representations: a study using eye-tracking technology. [20] Star, J.R., & Stylianides, G.J. 2013. Procedural and conceptual knowledge: Educational Research, 57(1), 59-79. exploring the gap between knowledge type and knowledge quality. Canadian [44] Mock, J., Huber, S., Klein, E., & Moeller, K. 2016. Insights into numerical Journal of Science, Mathematics and Technology Education, 13(2), 169-181. cognition: Considering eye-fixations in number processing and arithmetic. [21] Burton, R.F. 2001. Quantifying the effects of chance in multiple choice and Psychological Research, 80(3), 334-359. true/false tests: question selection and guessing of answers. Assessment & [45] Itti, L., & Koch, C. 2000. A saliency-based search mechanism for overt and Evaluation in Higher Education, 26(1), 41-50. covert shifts of visual attention. Vision Research, 40(10), 1489-1506. [22] Haffejee, F. & Sommerville, T.E. 2014. Fairness in using negative marking for [46] Mason, L., Tornatora, M.C., & Pluchino, P. 2013. Do fourth graders integrate assessing true/false questions. The Independent Journal of Teaching and text and picture in processing and learning from an illustrated science text? Learning, 9, 75-82. Evidence from eye-movement patterns. Computers & Education, 60(1), 95-109. [23] Schneider, M., Heine, A., Thaler, V., Torbeyns, J., De Smedt, B., Verschaffel, L., [47] Andrà, C., Arzarello, F., Ferrara, F., Holmqvist, K., Lindström, P., Robutti, O., & ... & Stern, E. 2008. A validation of eye movements as a measure of elementary Sabena, C. 2009. How students read mathematical representations: An eye school children's developing number sense. Cognitive Development, 23(3), 409- tracking study. Proceedings of the 33rd Conference of the International Group for 422. the Psychology of Mathematics Education (Vol. 2, pp. 49-56). Thessaloniki, [24] Covillion, J.D. 1995. Using number theory to reinforce elementary algebra. Greece, 19-24 July 2009. DOCUMENT RESUME, 17(1-2), 38. Retrieved from [48] Holmqvist, K., Andrà, C., Lindström, P., Arzarello, F., Ferrara, F., Robutti, O., & http://files.eric.ed.gov/fulltext/ED394533.pdf#page=42 (Accessed on 22 April Sabena, C. 2011. A method for quantifying focused versus overview behavior in 2014) AOI sequences. Behavior Research Methods, 43(4), 987-998. [25] Andrá, C., Lindström, P., Arzarello, F., Holmqvist, K., Robutti, O., & Sabena, C. [49] Krichevets, A.N., Shvarts, A.Y. & Chumachenko, D.V. 2014. Perceptual action 2015. Reading Mathematics representations: An eye-tracking study. of novices and experts in operating visual representations of a mathematical International Journal of Science and Mathematics Education, 13(2), 237-259. concept. Psychology. Journal of Higher School of Economics, 11(3), 55-78. [26] Storm, C.K., & Zullo, H. 2015. How can Mathematics students learn to play? [50] Lin, J.J., & Lin, S.S. 2014. Tracking eye movements when solving geometry Journal of Humanistic Mathematics, 5(1), 191-197. problems with handwriting devices. Journal of Eye Movement Research, 7(1):2, [27] Wall, J., Thompson, C.A., & Morris, B.J. 2015.Confidence judgments and eye 1-15. fixations reveal adults’ fractions knowledge. The annual meeting of the [51] Nugrahaningsih, N., Porta, M., & Ricotti, S. 2013. Gaze behavior analysis in cognitive science society. Retrieved from multiple-answer tests: An Eye tracking investigation. In Information Technology https://mindmodeling.org/cogsci2015/papers/0441/paper0441.pdf (Accessed Based Higher Education and Training (ITHET), 2013 International Conference on 4 September 2016) on (pp. 1-6, October). IEEE. [28] Godau, C., Haider, H., Hansen, S., Schubert, T., Frensch, P.A. & Gaschler, R. [52] Tai, R.H., Loehr, J.F., & Brigham, F.J. 2006. An exploration of the use of eye‐gaze 2014. Spontaneously spotting and applying shortcuts in arithmetic - a primary tracking to study problem‐solving on standardized science assessments. school perspective on expertise. Frontiers in Psychology. 5:556. doi International Journal of Research & Method in Education, 29(2), 185-208 10.3389/fpsyg.2014.00556 [53] Godau, C. 2015. Cognitive bases of spontaneous shortcut use in primary school [29] Nyström, M., & Ögren, M. 2012. How illustrations influence performance and arithmetic (Doctoral dissertation, Humboldt-Universität zu Berlin, eye movement behaviour when solving problems in vector calculus. In LTHs 7: Lebenswissenschaftliche Fakultät). Retrieved from e Pedagogiska Inspirationskonferens. http://edoc.hu-berlin.de/dissertationen/godau-claudia-2015-01- [30] Rozado, D., Agustin, J.S., Rodriguez, F.B., & Varona, P. 2012. Gliding and 12/PDF/godau.pdf (Accessed on 15 May 2016) saccadic gaze gesture recognition in real time. ACM Transactions on Interactive [54] Chesney, D.L., McNeil, N.M., Brockmole, J.R., & Kelley, K. 2013. An eye for Intelligent Systems (TiiS), 1(2), 10. relations: eye-tracking indicates long-term negative effects of operational [31] Anderson, J.R., Carter, C. & Koedinger, K.R. 2000. Tracking the course of thinking on understanding of math equivalence. Memory & Cognition, 41(7), mathematics problems. ROLE Award. Pittsburgh: Carnegie Mellon University. 1079-1095. Retrieved from [55] Cimen, O.A. 2013. Mathematics learner profiling using behavioral, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.103.9911 (Accessed physiological and self-reporting methods. Doctoral dissertation, Education: on 15 May 2014) Faculty of Education. Simon Fraser University. Retrieved from [32] Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Van http://summit.sfu.ca/item/13513 (Accessed on 15 March 2015) de Weijer, J. 2011. Eye tracking: A comprehensive guide to methods and measures. [56] Duchowski, A.T., 2007. Eye tracking methodology: Theory and practice. Second OUP Oxford. Edition. Springer. London, England. [33] Paruchuri, A. 2012. Think aloud: Can eye-tracking add value in detecting [57] Gronlund N.E. (1993). How to make achievement tests and assessments. 5th ed. usability problems? Thesis. Rochester Institute of Technology. Retrieved from New York, Mass: Simon & Shuster. http://scholarworks.rit.edu/theses/5751/ (Accessed on 15 May 2016) [58] Department of Basic Education. 2014. Grade 4 - Mathematics in English, Book 1, [34] Mason, L., Pluchino, P., Tornatora, M.C., & Ariasi, N. 2013. An eye-tracking Terms 1 & 2. Fourth edition. Pretoria. South Africa. study of learning from science text with concrete and abstract illustrations. The [59] Sottilare, R.A., Graesser, A., Hu, X., & Holden, H. (Eds.). 2013. Design Journal of Experimental Education, 81(3), 356-384. recommendations for intelligent tutoring systems: Volume 1-Learner Modeling [35] Gluck, K.A., Anderson, J.R., & Douglass, S.A. 2000. Broader bandwidth in (Vol. 1). US Army Research Laboratory. student modeling: What if ITS were “Eye” TS? In International Conference on [60] Tobii AB. 2014. Tobii Studio 3.3.0. Intelligent Tutoring Systems (pp. 504-513). Springer Berlin Heidelberg. [36] Lai, M.L., Tsai, M.J., Yang, F.Y., Hsu, C.Y., Liu, T.C., Lee, S.W.Y., ... & Tsai, C.C. 2013. A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational Research Review, 10, 90-115. [37] Mavrikis, M. 2016. A study on eye fixation patterns of students in higher education using an online learning system. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 408-416). Edinburgh, United Kingdom. 25-29 April 2016. ACM. [38] Hartmann, M., & Fischer, M.H. 2016. Exploring the numerical mind by eye- tracking: a special issue. Psychological Research, 80(3), 325-333. [39] Susac, A.N.A., Bubic, A., Kaponja, J., Planinic, M., & Palmovic, M. 2014. Eye movements reveal students’strategies in simple equation solving. International Journal of Science and Mathematics Education, 12(3), 555-577. [40] Ganor-Stern, D., & Weiss, N. 2016. Tracking practice effects in computation estimation. Psychological Research, 80(3), 434-448. [41] Van Gog, T., Paas, F., & Van Merriënboer, J.J. 2005. Uncovering expertise‐ related differences in troubleshooting performance: combining eye movement and concurrent verbal protocol data. Applied Cognitive Psychology, 19(2), 205- 221.