SYNESTHETIC COLOURS BASED ON VISUAL- AND PHONETIC GRAPHEME SIMILARITY DO NOT GIVE A SIGNIFICANT ADVANTAGE IN LEARNING THE ARABIC ALPHABET ON A SHORT TERM IN DUTCH NON-SYNESTHETES

Name student: Kelly Spaans Student number: 11700386

Supervisors: dr. R. Rouw, J. Gudden Examinator: dr. M. Otten

Date of submission: 29 January 2021

ABSTRACT

The Arabic language plays an essential role in world civilization; however, learning the Arabic script can be quite challenging. Colour coding methods seem to aid in this process, especially for grapheme synesthetes for which graphemes induce colour sensations. For learning a second language in adulthood, the similarity of the visual and phonetic properties to the first language seem to determine the transfer of synesthetic colours to these newly encountered graphemes. Though synesthesia is known to be a predisposed, fixed trait, recent studies have found a possibility for non-synesthetes to acquire synesthetic-like letter-colour associations that differ in colour palette from the non-synesthetic colour palette. In this study we focused on whether ‘synesthetic’ colouring of initial inducers from another alphabetic script (i.e., Modern Standard Arabic) affects the short-term learning process of this script for non- synesthetes. Twenty-one non-synesthetic native Dutch speakers executed a short-term memory task with a within-subject design in which they learned Arabic graphemes. We examined the learning effects of the synesthetic-coloured graphemes in contrast to black graphemes and whether there were any learning differences between colouring these graphemes based on visual or phonetic similarity with the Dutch alphabet. No significant short-term learning effect was found that resulted from synesthetically colouring the novel initial inducers. Moreover, no significant differences were found when these inducers were synesthetically coloured based on visual or phonetic properties. We hypothesize that these results are due to a missing learning curve that is caused by short-term grapheme-learning and an unstable letter-colour association. Therefore, more research into the advantages of synesthetic colouring in non-synesthetes is necessary. Follow-up studies should focus on long-term learning effects in terms of having a greater exposure time to the correct grapheme-sound association as well as the grapheme-colour association. We did not find an appropriate design to learn novel graphemes by using synesthetic colours based on different linguistic properties on short-term notice. However, there are still many possibilities of synesthetic colour-learning methods to explore that especially seem promising for long-term learning.

INTRODUCTION

With over 240 million native speakers, Arabic is the fifth most learned first language on a global scale (Lewis, Fennig, & Simons, 2009). Thus, it comes as no surprise that the Arabic language plays an essential role in world civilization: from science and commerce, to religion and influences on other cultures. In many European countries, including the Netherlands, the significance of the Arabic language resurfaced specifically with the arrival of refugees and asylum seekers. The majority of these individuals are native Arabic speakers and do not speak any other languages which often creates a language barrier with social workers and makes the integration procedure even more challenging and time-consuming. Though many education programmes currently focus on native Arabic refugees and asylum seekers learning the Dutch language, one should consider providing an education programme for Dutch social workers learning the Arabic language as well. This could lead to a better communication between the two parties and subsequently facilitate the integration process. Whether it is for business or social purposes, learning a new script such as the Arabic script can be quite challenging. Various methods to aid in this learning process have been researched, including colour coding which seems to have an advantage in encoding, storing, and retrieving stimuli (Dzulkifli & Mustafar, 2013). For novel languages in particular, colour coding has shown effects for Indonesian students learning English vocabulary (Nurdiansyah, Asyid, & Parmawati, 2019), children learning modified Hebrew letters (Goodman & Cundick, 1976), and for children discriminating unfamiliar words and letters (Jones, 1965). Furthermore, cases where colours seem to be exceptionally helpful in lexical learning often include grapheme-synesthetes.

Synesthesia can be described as a perceptual phenomenon in which a stimulus, the inducer, can induce extraordinary sensations, known as concurrents (Root et al., 2018). The inducer and its subsequent concurrent are often from different modalities (Uno, Asano, Kadowaki, & Yokosawa, 2020), making this phenomenon intriguing from a neurological perspective. A large number of studies have therefore focused on structural and functional brain differences that could cause these types of strong associations. Compared to non-synesthetes, synesthetes showed structural brain differences in white and grey matter properties (Rouw & Scholte, 2007) and altered activation patterns within specific regions of the brain (Sinke et al., 2012). The specific differences are often dependent of the type of synesthesia, as synesthetic experiences can occur between a variety of different modalities. The most common form of synesthesia that includes the visual modality is grapheme synesthesia. In this condition, visual perception of graphemes induce colour sensations (Uno et al., 2020). A model that focuses on structural differences for grapheme synesthesia has been proposed by Hubbard & Ramachandran (2005). This model describes the synesthetic experiences (i.e., perception of colour) as a result of cross-activation, ‘hyperconnections’, within the fusiform gyrus for ‘grapheme’ and ‘colour’ regions.

Apart from research into the neurological mechanisms of synesthesia, studies have focused on the psychological effects as well, such as memory advantage. The results from different studies seem to be ambivalent: On the one hand, there are a number of individual cases where exceptional memory seems to be caused by synesthetic experiences (Mills, Innis, Westendorf, Owsianiecki, & McDonald, 2006; Smilek, Dixon, Cudahy, & Merikle, 2002). In a review of Watson, Akins, Spiker, Crawford, & Enns (2014) these advantages are explained in the sense that synesthetes can use their concurrents to obtain and retain information of the initial inducers. On the other hand, based on group studies such as Rothen & Meier (2010), the idea of an ‘extraordinary’ memory advantage is often criticized. Although extraordinary effects have limited evidence, it can be cautiously concluded that there is some sort of memory advantage in the form of ‘superior encoding’ for synesthetes in their specific inducer-concurrent domain (Gross, Neargarder, Caldwell-Harris, & Cronin-Golomb, 2011; Watson et al., 2014). For grapheme synesthetes, the use of colour can be particularly helpful for learning novel graphemes (Blair & Berryhill, 2013).

Synesthesia is known to be a predisposed, fixed trait. However, recent studies have found a possibility for non-synesthetes to acquire a subset of certain synesthetic behavioural traits through a training that involved reading in colour (Colizoli, Murre, & Rouw, 2012). The key traits that describe (grapheme) synesthesia are the automatic occurrence in which the sensations appear and their consistency over time (Grossenbacher & Lovelace, 2001) In regard to the concurrents, the colour choices in grapheme synesthetes are often idiosyncratic. However, the full ‘synesthetic’ colour palette is very specific and differs from regular non- synesthetic letter-colour associations (Rouw & Root, 2019). Moreover, there seems to be a certain influence of linguistic properties such as visual form, sound, semantics, and letter frequency on colour choices for specific graphemes (Asano & Yokosawa, 2011). For learning a second language in adulthood, the similarity of the visual and phonetic properties to the first language seem to determine the transfer of synesthetic colours to these newly encountered graphemes (Asano & Yokosawa, 2011; Witthoft & Winawer, 2006). However, the extent to which both of these properties have an impact on this ‘transfer’ varies across writing systems (Asano & Yokosawa, 2013). Transmission of synesthetic colours have been investigated for Glagolitic (Mroczko, Metzinger, Singer, & Nikolić, 2009), Hebrew, Devanagari, Armenian, Cyrillic (Blair & Berryhill, 2013), and even for logographic units (Simner, Hung, & Shillcock, 2011), but not yet for the Arabic language.

The Arabic language is closely related to Hebrew, Aramaic, and Amharic (Ryding, 2005). Its spoken form contains multiple varieties of which Egyptian Arabic is used most commonly (Van Leeuwen, Dingemanse, Todil, Agameya, & Majid, 2016). As regards to the written form, Modern Standard Arabic (MSA) is considered the only official form of Arabic and is therefore used mostly in written documents (Van Leeuwen et al., 2016). The Arabic script, written left to right, consists of 28 characters that, independently, mostly represent consonant sounds, with a few semivowels and a few long vowels (Ryding, 2005). Short vowels can optionally be indicated using diacritics above or below the initial characters. however, these are often omitted in MSA (Van Leeuwen et al., 2016). Dependent of their position in a word, or whether they are isolated, the characters take on a different form (i.e., a final, middle, initial, or independent form) (Ryding, 2005).

Considering the fact that colours can aid in language learning, synesthetic associations can be acquired by non-synesthetes through training, and the fact that the obtaining and retaining of the initial inducers (i.e., graphemes) are enforced by this type of synesthetic colour- letter associations, it would be interesting to investigate whether ‘synesthetic’ colouring of initial inducers from another alphabetic script (e.g., MSA) affects the short-term learning process of this script for non-synesthetes. Moreover, it would be even more interesting to take different linguistic properties into account for the colouring of these inducers. Therefore, we will attempt to answer the following research question: do synesthetic colours based on visual and phonetic similarity give a significant advantage in learning the Arabic alphabet on a short term in Dutch non-synesthetes? Furthermore, are there any significant differences in learning effects between visual and phonetic similarity-based methods for synesthetic colouring?

This study distinguished two linguistic properties (visual and phonetic) that were used as a manipulation of the synesthetic colour that was assigned to the graphemes. Thus, in the experimental conditions the Arabic graphemes were ‘synesthetically’ coloured based on their shape- or sound similarity with Dutch graphemes. In the control condition, the Arabic graphemes were coloured to a default colour (black). Using a short-term memory task with a within-subject design, participants learned novel Arabic graphemes and their corresponding phoneme that were coloured dependent of the condition. We examined the learning effects of the synesthetic-coloured graphemes and whether there were any learning differences between colouring these graphemes based on visual or phonetic similarity with the Dutch alphabet. We predicted that both ‘synesthetic coloured’ conditions would improve the participant’s performance on the learning task for Arabic graphemes and phonemes compared to the ‘default’ condition in which the letters were coloured black. As earlier research found that phonetic similarity with the first language seems to be the strongest factor to determine the transfer of synesthetic colours when novel graphemes differ greatly in visual form with the first language and this already applies for the closely related Hebrew script (Blair & Berryhill, 2013), we predicted that the sound-similarity condition will have a greater benefit on performance than the visual-similarity condition.

MATERIALS AND METHOD

Participants

Twenty-one native Dutch students without any forms of synesthesia (sixteen females, five males, mean age = 21.8 years, SD = 1,25 years) were recruited for this study. The participants had normal or corrected-to-normal vision and hearing and reported no knowledge of the Arabic language or the Arabic script. Other exclusion criteria were dyslexia and colour blindness. Furthermore, participants reported knowledge of other languages, namely English (100%), German (42,9%), French (19,0%), Spanish (14,3%), and Russian (4,8%). The experiment was approved by the Ethics Committee of the University of Amsterdam and all participants gave informed consent prior to the experiment. If applicable, the participants were compensated with 1.00 Research Credit (RC).

Stimuli

The stimuli consisted of Arabic graphemes that were presented both visually and auditory for each trial. The auditory presentation of the grapheme contained both the spoken Arabic grapheme name and the corresponding sound, whereas the visual presentation included the Arabic grapheme that was coloured dependent of the condition. Since the synesthetic colour palette for the Modern Arabic script by Dutch participants has not been discovered yet, the Arabic graphemes were coloured based on their visual or auditive similarity with Dutch graphemes. In other words, the Dutch visual or sound ‘equivalent’ determined which synesthetic colour was applied to the Arabic grapheme used in the experiment. Synesthetic colours for Dutch graphemes were derived from a databased created by Rouw & Root (2019). If the synesthetic colours were the same for both experimental conditions (i.e., based on shape and sound), the concerning Arabic grapheme was eliminated from the stimulus set. Other reasons of elimination and the procedure prior to picking colours for the experimental conditions will be elucidated in the next sections.

Colours based on shape

In this condition Arabic graphemes were coloured based on their shape similarity with Dutch graphemes. Similarity was determined based on Gibson’s Shape Classification System (Geyer & DeWald (1973) for Dutch graphemes, and Wiley, Wilson, & Rapp (2016) for Arabic graphemes). In this system, graphemes are defined by the presence or absence of various visual features. The more shared visual features between the graphemes, the higher the final similarity score. The synesthetic colour of the Dutch grapheme with the highest similarity score was used for the corresponding Arabic grapheme. For example, when comparing visual features between and the Dutch alphabet (capitalized and non-capitalized), the Dutch ’ا‘ the Arabic grapheme was coloured according to ’ا‘ grapheme ‘I’ elicited the highest similarity score and therefore the synesthetic colour for ‘I’, which in this case was white. An example of the scheme used to determine similarity scores is shown in Fig. 1.

and Roman capital letters. All light green ‘ ب‘ Fig. 1: A shape similarity scheme for the Arabic character as seen in the blue box in the left corner ,‘ ب‘ boxes indicate a congruence between the Arabic character above, and a Roman capital letter. The total score is calculated by the obtained ‘congruence points’, here presented as the row total of green boxes. One point equals one congruent distinctive feature. The Roman letter coloured yellow depicts the most congruent letter, here marked for ‘U’.

Arabic graphemes were compared to both lowercase and capital letters in the default ‘Calibri’ font, except for the letter ’g’, which was used in the ‘Arial’ font ‘g’ as well. The highest similarity scores ranged between 9 and 12 (out of 12). For some Arabic graphemes, multiple Dutch graphemes evoked the highest similarity score compared to other Dutch graphemes. In this case, the final Dutch grapheme used for synesthetic colouring was chosen based on a more is most similar to lowercase ’ط‘ specific feature analysis. For example, the Arabic grapheme letters ‘b’, ‘d’, ‘q’, ‘p’ and capital letter ‘P’. However, when looking more specifically at features such as ‘mirroring’, the letter ‘b’ fitted best and was therefore used for the synesthetic colouring of the Arabic grapheme in the experiment for the ‘shape’ condition. In addition to more specific analyses, letters were chosen based on how well they fit the experiment. For example, based on the frequency of colours, one letter might fit the experiment better than another letter. Moreover, if the letter turned out to elicit the synesthetic colour ‘black’, this letter was not chosen. All Arabic graphemes that only had the option to elicit a black letter were removed from the stimulus set, otherwise they would overlap with the control condition. Furthermore, elimination of an Arabic grapheme could occur when the grapheme was removed in the ‘auditive similarity condition’, as seen in the next section.

Colours based on sound

This condition contained Arabic graphemes that were coloured based on their Dutch contains a voiceless alveolar fricative that ’ ص ‘ sound equivalent. For example, the letter was ’ ص‘ corresponds with the sound / s / (Ryding, 2005). Therefore, in the experiment coloured red, corresponding to the synesthetic colour for ‘S’. This method works properly for most Arabic graphemes. However, there are twelve letters in the Arabic alphabet that have a complex sound or a sound that is non-existent in the Dutch language. Therefore, for some letters we selected a Dutch phoneme that is not identical, but clearly comparable to the Arabic sound. Other letters were selected based on comparable English or French phonemes. For example, the with the sound /d͡ ʒ/ can be coloured according to the English phoneme ‘j’, as seen in ج letter containing a /ɣ/ sound can be coloured according to the French phoneme غ jeep’, and the letter‘ ‘r’, as seen in the French pronunciation of ‘Paris’. Given the fact that all participants were Western-European, we assumed that these sounds were known and could be used for the colouring procedure. The comparisons between phonemes for different languages were based on the International Phonetic Alphabet (IPA) (Smith, 2000). The choice of grapheme for synesthetic colouring in the ‘sound’ condition for each Arabic letter is shown in Fig. 2.

Figure 2. Choosing the grapheme for synesthetic colouring based on (close) Dutch, English, and French equivalent sounds. Twenty-eight Arabic graphemes (first column) were coloured according to the letter (third column) that had a similar sound (second column) to either Dutch phonemes (blue), closely related Dutch phonemes (cyan), (close) English phonemes (green), or a single French phoneme (yellow).

were represented by phonemes that contained more than two ث , خ , ذ , ش , ظ The letters letters when translating these phonemes to the sounds of the English or Dutch language. For example, /th/ and /sj/ consist of ‘t’ and ‘h’, and ‘s’ and ‘j’, which makes it difficult to firmly choose one letter for colouring. Thus, these letters were eliminated from the experiment.

Based on these conditions and the conditions mentioned in the previous section regarding shape similarity, a total of ten letters had been eliminated for the experiment, which gave an overall stimulus set of eighteen Arabic graphemes with an appropriate shape- or phoneme-equivalent to which the graphemes were coloured. The final stimulus set is shown in Fig. 3.

Fig. 3: Final stimulus set that was used in the experiment. From left to right: the stimulus set with Arabic graphemes that were synesthetically coloured according to their sound similarity for the ‘sound’ condition, the stimulus set with Arabic graphemes that were synesthetically coloured according to their shape similarity for the ‘shape’ condition, and the stimulus set with Arabic graphemes that had a ‘default’ colour for the control condition.

Procedure

Each participant completed the experiment online, at a self-chosen location, in Qualtrics (Qualtrics, Provo, UT). The experiment started with the informed consent, followed by a few auditory test trials with Dutch words to test for proper audio quality, a questionnaire, a learning phase, a testing phase, and a few final questions at the end of the experiment about the participant’s experience during the experiment and possible strategies that were used to memorize the grapheme-sound association (Fig. 4). The experiment had a within-subject design. To correct for possible confounds, the conditions within trials and the order of presented graphemes were randomized and counterbalanced between groups of participants.

Fig. 4: Flowchart of the experimental design. The experiment started with the informed consent. If the participant did not give consent, they were directed to the debrief. When consent was given, the experiment started with an auditory test, (demographic) questions, a learning phase for words (light blue) and graphemes (dark blue), a testing phase for graphemes (dark blue), words (light blue), and a colour retrieval test (white). The experiment ended with a few final questions.

Questionnaire

After giving consent and completing an auditory test, the participants were asked a few simple demographic questions such as gender, age, and educational level. Subsequently, the questionnaire continued with questions about possible synesthetic letter-colour experiences in daily life, other forms of synesthesia, visual acuity, hearing, dyslexia, and the degree of affinity with the Arabic alphabet and other alphabets or languages.

Learning phase

The first part of the learning phase focused on Arabic words. After a brief instruction, 21 Arabic words consisting of four letters with the corresponding Dutch translation were each displayed for five seconds. Participants were instructed to remember the word-translation combination. The words were not supported aurally, and all words were presented in a default colour, namely black.

This word-learning phase was followed by a learning phase for the Arabic graphemes. As described earlier, graphemes were displayed in a fixed random order and a fixed random condition, this was counterbalanced between participants. In total, eighteen graphemes were presented to the participant that were either coloured based on shape, based on sound, or to a default colour. The presentation of the graphemes was supported aurally by an audio fragment which contained the name of the letter (e.g., ‘alif’), and its sound (/aː/). The audio fragments were derived from the Arabic Reading Course (Big Languages, 2021). Participants were instructed to remember the grapheme-sound combination. In addition to remembering this combination, participants had to indicate on a scale of 0-100 how confident they were of remembering the grapheme-sound association by clicking on the scale. As the confidence scale was part of a larger study and was not relevant for our research question, we did not analyse these results. After completing the trial, participants were free to click to proceed to the next trial until they had run through all eighteen trials. See Fig. 5 for an example of a learning trial for a grapheme.

Fig. 5: Grapheme learning trial. In this figure, the Arabic grapheme is presented with an audio fragment (in the top) that contains the spoken name and sound. The grapheme is coloured according to its condition, which in this case is the ‘shape’-condition. A 1-100 confidence scale that the participants manipulated can be found in the bottom of this figure.

Testing phase

Next, the participants entered the testing phase that started with a grapheme test with feedback. After reading the instruction, participants were presented with an audio fragment that corresponded to one of the four Arabic graphemes that were shown on the participant’s screen. The four graphemes were coloured according to their assigned condition and the distractors were chosen based on a high similarity of sound and shape with the correct grapheme to manage a steady level of difficulty. The participants were asked to choose the grapheme that corresponded with the given audio fragment (Fig. 6). After choosing one of the four options, the participant was directed to a screen where they got feedback on the trial (‘correct’ or ‘incorrect’). The correct grapheme-sound association was shown as well. After playing the audio fragment, the participant could move on to the next trial, until they had reached a total of eighteen trials. The second part of the testing phase for graphemes was similar to the first part, except for the feedback afterwards, this was omitted here. Subsequently, the participants started the testing phase of 21 trials for the learned Arabic words. The translation was presented with four options of Arabic letter combinations underneath, one of which was correct. After choosing one of the four options, the participant was directed to the next trial without any feedback on the completed trial. The word learning and testing phase served as a distractor task in this study, therefore we did not analyse these results. The third part of the testing phase involved further testing of the Arabic graphemes, however, the presented graphemes had no colour (default) and no feedback was given here either. Finally, the participants were asked to try to reproduce the colour each grapheme was presented in by manipulating a colour wheel. In addition to this, they were asked how confident they were of their choice on a scale of 1-100. After giving their colour indication for every grapheme, the participants were led to the final questions and the debrief.

Fig. 6: Grapheme testing trial. In this figure, the audio file containing the spoken name and sound is shown at the top. Underneath are the four grapheme options, each of them coloured according to their condition. Participants could choose one of the four options that they thought corresponded to the presented audio file.

RESULTS

A number of 45 subjects was recruited of which 21 did not complete the experiment and were thus removed from the dataset. Of the remaining subjects, three indicated that they had dyslexia. In accordance with the established exclusion criteria, these subjects were removed from the dataset as well. Next, we looked at subject outliers by means of calculating the interquartile range (IQR) and subsequently setting a lower and upper bound for the overall performance scores (LB = 4.0; UB = 46). Two subjects were just above the upper bound (n = 2; total score = 47). Nevertheless, we decided to include these subjects anyway because when we looked at the data, the participants showed serious dedication that conceivably led to higher scores rather than that they did by means of cheating. This resulted in a total number of 21 participants that had an overall subject score mean of 32.2 with a standard deviation of 9.29 (see Table 1 for demographic data and subject scores). For individual outliers (graphemes), another IQR was computed and used to set the lower and upper bound (LB = 8.5; UB = 52). Only the crossed the upper bound (n = 1; total score = 54). As we assumed this ’ا‘ Arabic grapheme particular outlier would not interfere with our results significantly, and to preserve all grapheme data, we decided to keep this grapheme in the dataset.

In order to test our research question with regards to the relationship between accuracy on the Arabic alphabet task, and the condition and testing phase the participant was in, a logistic mixed effect regression model was fitted to the obtained data. Here, accuracy was computed by subject scores (i.e., correct answers per condition per testing phase). The interaction between the Testing Phase (test with feedback, test without feedback, and test without feedback and colour) and the Condition (sound, shape, and black) were entered into a model as fixed effects. A subject variable was entered as a random effect. To test for significance of the Condition coefficient on accuracy, a ‘partial’ model was computed in which the Condition variable was dropped. The exact significance was calculated by comparing the two models using a Likelihood Ratio Test (LRT). For a visual approximation of the effect sizes, see Fig. 7. To test for a possible learning effect, a ‘null’ model was constructed in which both the Testing Phase and Condition variable were removed. This model was then compared to the ‘partial’ model and an LRT was used here as well to determine significance.

The statistical analysis was conducted in the software program R (R Core Team, 2014), using the lme4 package (Bates, Maechler, Bolker, & Walker, 2015). The LRT on the full and partial model suggests that there was no significant effect of Condition on accuracy (χ2 = 4.080, p = .667). For the ‘partial’ and ‘null’ model, the LRT did not result in any significance either (χ2 = 3.273, p = .195), indicating that there was no significant effect for learning over time. Lastly, we wanted to test for a possible correlation between accuracy for subject scores and accuracy for colour retrieval (measured by CIELuv distances). As this data did not meet the normality assumption and the non-parametric test did not result in any significance, we did not perform any further statistical tests on this data.

Table 1a. Demographic data. This table shows the demographic data of all participants (n = 21), including sex (sixteen females, five males), age (mean = 21.8 years, SD = 1,25 years), highest completed education level, reports of grapheme synesthesia, and other possible forms of synesthesia.

Participant Sex Age Highest completed Grapheme Other possible forms of education level¹ synesthesia synesthesia 1 Female 21 VWO No Days, months, and years have a colour ('maybe') 2 Female 24 HAVO No None 3 Female 22 VWO No Letters have personalities ('maybe'), 4 Female 22 Bsc No None 5 Male 25 Bsc No Numbers have colours ('maybe') 6 Female 21 HBO No Days, months, and years have a location in space ('maybe'); Numbers have a location in space ('yes') 7 Female 21 VWO No Days, months, and years have a colour ('maybe') 8 Female 20 VWO No Letters have personalities ('maybe'); Days, months, and years have a location in space ('yes'); Numbers have a location in space ('yes'); Days, months, and years have a colour ('maybe') 9 Female 22 HAVO No None 10 Female 20 VWO No Days, months, and years have a location in space ('maybe') 11 Male 22 MBO No None 12 Female 22 Bsc No Days, months, and years have a location in space ('yes'); Numbers have a location in space ('yes') 13 Male 21 Bsc No None 14 Female 23 Bsc No None 15 Male 22 HBO No Letters have personalities ('maybe') 16 Female 22 Bsc No Days, months, and years have a colour ('yes') 17 Male 20 Bsc No Letters have genders ('maybe'); Letters have personalities ('maybe'); Days, months, and years have a colour ('maybe') 18 Female 23 VWO No Letters have genders ('maybe') 19 Female 21 VWO No None 20 Female 22 VWO No Days, months, and years have a location in space ('yes') 21 Female 22 Bsc No None 1: Abbreviations for Dutch education levels: HAVO, Hoger Algemeen Voortgezet Onderwijs; VWO, Voortgezet Wetenschappelijk Onderwijs; MBO, Middelbaar Beroepsonderwijs; HBO, Hoger Beroepsonderwijs; Bsc, Bachelor’s degree.

Table 1b. Subject test scores. Scores are displayed for each condition within each test and a total score of the regarding test. The last column illustrates overall subjects scores (mean = 32.2; SD¹ = 9.29; IQR¹ = 14.0).

Test Test without Test with Overall without feedback feedback score feedback and colour Participant Correct Correct Correct Total Correct Correct Correct Total Correct Correct Correct Total Total sound shape black correct sound shape black correct sound shape black correct correct 1 2 3 2 7 3 4 1 8 2 1 2 5 20 2 3 4 4 11 5 4 5 14 5 4 5 14 39 3 2 4 4 10 5 4 6 15 3 3 6 12 37 4 1 4 6 11 4 4 4 12 5 4 5 14 37 5 3 3 3 9 2 3 3 8 3 3 2 8 25 6 2 3 4 9 2 4 3 9 1 2 4 7 25 7 6 4 4 14 5 6 5 16 6 6 5 17 47 8 4 2 5 11 4 4 5 13 4 3 5 12 36 9 1 2 3 6 3 2 3 8 2 2 3 7 21 10 1 4 3 8 2 3 4 9 2 4 4 10 27 11 3 2 1 6 2 0 2 4 2 0 3 5 15 12 4 3 4 11 4 3 4 11 4 2 3 9 31 13 2 4 3 9 4 3 4 11 2 3 3 8 28 14 3 3 1 7 1 2 3 6 4 3 1 8 21 15 5 5 5 15 5 5 5 15 6 5 4 15 45 16 5 3 4 12 5 5 5 15 5 6 5 16 43 17 5 4 6 15 5 6 5 16 5 6 5 16 47 18 3 5 1 9 4 4 4 12 3 4 4 11 32 19 4 2 3 9 3 5 4 12 3 4 4 11 32 20 3 4 6 13 3 6 3 12 4 5 5 14 39 21 3 5 2 10 3 2 2 7 5 3 4 12 29 1: Abbreviations – IQR: Inter Quartile Range; SD: Standard Deviation.

Fig. 7: Plotted confidence intervals on the coefficients of the fixed effects in the compared ‘full’ and ‘partial’ model based on their standard errors by using a normal approximation. The coefficient is the change in log-odds associated with a change in group membership. As seen in the plot, the confidence intervals cross zero, thus the effect of the Condition coefficient on accuracy tends to be statistically insignificant.

DISCUSSION

The purpose of this study was to investigate whether synesthetic colouring of novel initial inducers could affect the process of short-term learning of these novel inducers in non- synesthetes. Furthermore, we were interested in whether there were any differences in learning effects between visual and phonetic similarity-based methods for this synesthetic colouring. As is indicated by the obtained data, we found no significant short-term learning effect that resulted from synesthetically colouring the novel initial inducers. Moreover, no significant differences were found when these inducers were synesthetically coloured based on visual or phonetic properties.

From these results we can cautiously conclude that synesthetic colouring, whether this is based on similar visual or phonetic properties, does not seem to have an advantage in the process of short-term learning of novel Arabic initial inducers for Dutch non-synesthetes. Interestingly enough, regardless of colouring, we found no learning effects at all which makes finding other effects such as synesthetic colouring more challenging. In other words, the experimental design that was used in this research was too insufficient to form a substantial learning curve that supports other potential effects. When looking at the design, we can find a few possible explanations for this problem. First of all, the length of the learning phase was quite short: all graphemes were shown once and the total amount of time for this part of the experiment lasted no longer than eight minutes. Likewise, the testing phase only had one part in which feedback was given, this means that the total amount of times that the participants were exposed to the correct grapheme-sound association was only twice.

Another noteworthy aspect of our study is the element of testing short-term memory. Considering previous studies in which non-synesthetes managed to acquire synesthetic-like letter-colour associations after a long term of exposure (Colizoli et al., 2012), a short-term approach used in this experiment might have been the pitfall that led to the missing initial learning effect and the lack of variance between coloured and non-coloured learned graphemes. In addition to this, when we consider the difficulty of learning Arabic graphemes in general, long-term exposure could be necessary as well to actually establish a learning effect for these graphemes before finding any other effects. All in all, synesthetic colours might be helpful for non-synesthetes, but a long-term exposure to these colours and the novel graphemes should be considered first.

Subjective descriptions of strategies could be seen as evidence for the previous statement. To one of the ending questions that asked whether the participant had used any strategies, answers often involved the description of strategies where shapes were made up from the Arabic graphemes as a mnemonic for learning these novel graphemes. For example, one ,’as ‘the sceptre of Moses ’م‘ participant described that they remembered the Arabic grapheme in which the letter ‘M’ was the corresponding letter to the Arabic grapheme. This type of shape- related mnemonic strategy might overload the mnemonic strategy that involves the use of synesthetic colours. Therefore, finding an effect for colouring could be very unlikely, because it has the ‘confound’ of this shape-related mnemonic strategy. Again, long-term grapheme- colour exposure leads to a more strong and stable association and this might eventually take the upper hand in terms of choosing the synesthetic colour-strategy.

Another possible limitation in regard to the experimental design is the use of an objective measure to determine which Dutch grapheme is most similar to the concerning Arabic grapheme. While sound has a more or less one-to-one relationship with a Dutch letter, the shape-similarity of the Arabic grapheme can be interpreted quite broadly by different subjects which can lead to a ‘wrongly’ chosen synesthetic colour for that grapheme. One could solve this by extending the design and apply the subject’s own interpretation into the design for the shape-coloured graphemes. For example, the participant indicates to think of the Dutch letter Therefore, in the next section of the experiment .ش W’ when seeing the Arabic grapheme‘ where the participant will start the learning- and test procedure, the synesthetic colour for ‘W’ is applied for all graphemes that are synesthetically coloured by shape. Furthermore, there is a possibility that some Arabic graphemes do not elicit any similar Dutch graphemes for the participant. This could be settled by asking how strong the resemblance was on a scale of 1- 100 (‘Please indicate how strong the resemblance for the chosen Dutch grapheme is from a scale of 1-100’).

All in all, more research into the advantages of synesthetic colouring in non- synesthetes is necessary. However, this study gives a decent push towards finding methods that make learning a novel alphabet easier, but mostly more robust, by using synesthetic colours. Further research is necessary to correct for the limitations of the current study. Therefore, we suggest that follow-up studies should focus on long-term learning effects in terms of having a greater exposure time to the correct grapheme-sound association as well as the grapheme-colour association. To avoid ceiling effects for the learning of novel graphemes, subjects without synesthesia could form synesthetic-like letter-colour associations in their own language first before learning novel graphemes that are synesthetically coloured based on similar linguistic properties with their first language. In other words, non-synesthetes would eventually ‘imitate’ synesthetes in the transfer of the synesthetic colours to a new script and use their synesthetic- induced traits to aid them in remembering the Arabic, or any other alphabet. Furthermore, further research should look closely at the experimental design and apply subjective interpretations in the experiment. If the right adjustments are made, new synesthetic-colouring methods to possibly aid in learning a new alphabet could be developed in the future.

In conclusion, finding an appropriate design to learn novel graphemes by using synesthetic colours based on different linguistic properties can be quite challenging on short- term notice. However, there are still many possibilities of synesthetic colour-learning methods to explore that especially seem promising for long-term learning.

REFERENCES

Asano, M., & Yokosawa, K. (2011). Synesthetic colors are elicited by sound quality in Japanese synesthetes. Consciousness and Cognition. https://doi.org/10.1016/j.concog.2011.05.012 Asano, M., & Yokosawa, K. (2013). Grapheme learning and grapheme-color synesthesia: Toward a comprehensive model of grapheme-color association. Frontiers in Human Neuroscience. https://doi.org/10.3389/fnhum.2013.00757 Bates, D. M., Maechler, M., Bolker, B., & Walker, S. (2015). lme4: linear mixed-effects models using S4 classes. Journal of Statistical Software. Big Languages (2021). Arabic Reading Course. http://arabicreadingcourse.com/learn-the- arabic-alphabet.php Blair, C. D., & Berryhill, M. E. (2013). Synesthetic grapheme-color percepts exist for newly encountered Hebrew, Devanagari, Armenian and Cyrillic graphemes. Consciousness and Cognition. https://doi.org/10.1016/j.concog.2013.06.002 Colizoli, O., Murre, J. M. J., & Rouw, R. (2012). Pseudo-Synesthesia through reading books with colored letters. PLoS ONE. https://doi.org/10.1371/journal.pone.0039799 Dzulkifli, M. A., & Mustafar, M. F. (2013). The influence of colour on memory performance: a review. The Malaysian Journal of Medical Sciences : MJMS. Geyer, L. H., & DeWald, C. G. (1973). Feature lists and confusion matrices. Perception & Psychophysics. https://doi.org/10.3758/BF03211185 Goodman, M. D., & Cundick, B. P. (1976). Learning rates with black and colored letters. Journal of Learning Disabilities. https://doi.org/10.1177/002221947600900912 Gross, V. C., Neargarder, S., Caldwell-Harris, C. L., & Cronin-Golomb, A. (2011). Superior encoding enhances recall in color-graphemic synesthesia. Perception. https://doi.org/10.1068/p6647 Grossenbacher, P. G., & Lovelace, C. T. (2001). Mechanisms of synesthesia: Cognitive and physiological constraints. Trends in Cognitive Sciences. https://doi.org/10.1016/S1364- 6613(00)01571-0 Hubbard, E. M., & Ramachandran, V. S. (2005). Neurocognitive mechanisms of synesthesia. Neuron. https://doi.org/10.1016/j.neuron.2005.10.012 JONES, J. K. (1965). COLOUR AS AN AID TO VISUAL PERCEPTION IN EARLY READING. The British Journal of Educational Psychology. https://doi.org/10.1111/j.2044-8279.1965.tb01783.x Lewis, M. P., Fennig, C. D., & Simons, G. F. (2009). Ethnologue: Languages of the World, Sixteenth edition. SIL International. Mills, C. B., Innis, J., Westendorf, T., Owsianiecki, L., & McDonald, A. (2006). Effect of a synesthete’s photisms on name recall. Cortex. https://doi.org/10.1016/S0010- 9452(08)70340-X Mroczko, A., Metzinger, T., Singer, W., & Nikolić, D. (2009). Immediate transfer of synesthesia to a novel inducer. Journal of Vision. https://doi.org/10.1167/9.12.1 Nurdiansyah, D. M. R., Asyid, S. A., & Parmawati, A. (2019). USING COLOR CODING TO IMPROVE STUDENTS’ ENGLISH VOCABULARY ABILITY. PROJECT (Professional Journal of English Education). https://doi.org/10.22460/project.v2i3.p358- 363 R Core Team. (2014). R Core Team (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL Http://Www.R-Project.Org/. Root, N. B., Rouw, R., Asano, M., Kim, C. Y., Melero, H., Yokosawa, K., & Ramachandran, V. S. (2018). Why is the synesthete’s “A” red? Using a five-language dataset to disentangle the effects of shape, sound, semantics, and ordinality on inducer–concurrent relationships in grapheme-color synesthesia. Cortex. https://doi.org/10.1016/j.cortex.2017.12.003 Rothen, N., & Meier, B. (2010). Grapheme-colour synaesthesia yields an ordinary rather than extraordinary memory advantage: Evidence from a group study. Memory. https://doi.org/10.1080/09658210903527308 Rouw, R., & Root, N. B. (2019). Distinct colours in the “synaesthetic colour palette.” Philosophical Transactions of the Royal Society B: Biological Sciences. https://doi.org/10.1098/rstb.2019.0028 Rouw, R., & Scholte, H. S. (2007). Increased structural connectivity in grapheme-color synesthesia. Nature Neuroscience. https://doi.org/10.1038/nn1906 Ryding, K. C. (2005). A reference Grammar of modern standard Arabic. A Reference Grammar of Modern Standard Arabic. https://doi.org/10.1017/CBO9780511486975 Simner, J., Hung, W. Y., & Shillcock, R. (2011). Synaesthesia in a logographic language: The colouring of Chinese characters and Pinyin/Bopomo spellings. Consciousness and Cognition. https://doi.org/10.1016/j.concog.2011.05.006 Sinke, C., Neufeld, J., Emrich, H. M., Dillo, W., Bleich, S., Zedler, M., & Szycik, G. R. (2012). Inside a synesthete’s head: A functional connectivity analysis with grapheme- color synesthetes. Neuropsychologia. https://doi.org/10.1016/j.neuropsychologia.2012.09.015 Smilek, D., Dixon, M. J., Cudahy, C., & Merikle, P. M. (2002). Synesthetic color experiences influence memory. Psychological Science. https://doi.org/10.1111/1467-9280.00496 Smith, C. L. (2000). Handbook of the International Phonetic Association: A guide to the use of the International Phonetic Alphabet (1999). Phonology. https://doi.org/10.1017/S0952675700003894 Uno, K., Asano, M., Kadowaki, H., & Yokosawa, K. (2020). Grapheme-color associations can transfer to novel graphemes when synesthetic colors function as grapheme “discriminating markers.” Psychonomic Bulletin and Review. https://doi.org/10.3758/s13423-020-01732-9 Van Leeuwen, T. M., Dingemanse, M., Todil, B., Agameya, A., & Majid, A. (2016). Nonrandom associations of graphemes with colors in Arabic. Multisensory Research. https://doi.org/10.1163/22134808-00002511 Watson, M. R., Akins, K. A., Spiker, C., Crawford, L., & Enns, J. T. (2014). Synesthesia and learning: A critical review and novel theory. Frontiers in Human Neuroscience. https://doi.org/10.3389/fnhum.2014.00098 Wiley, R. W., Wilson, C., & Rapp, B. (2016). The effects of alphabet and expertise on letter perception. Journal of Experimental Psychology: Human Perception and Performance. https://doi.org/10.1037/xhp0000213 Witthoft, N., & Winawer, J. (2006). Synesthetic colors determined by having colored refrigerator magnets in childhood. Cortex. https://doi.org/10.1016/S0010- 9452(08)70342-3