Grapheme–Color Synesthesia Influences Overt Visual Attention Jonathan S. A. Carriere1, Daniel Eaton1, Michael G. Reynolds2, Mike J. Dixon1, and Daniel Smilek1 Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/2/246/1759957/jocn.2009.21048.pdf by guest on 18 May 2021 Abstract & For individuals with grapheme–color synesthesia, achromatic exhibited a color congruity bias—a propensity to fixate con- letters and digits elicit vivid perceptual experiences of color. gruently colored letters more often and for longer durations than We report two experiments that evaluate whether synesthesia incongruently colored letters—in a naturalistic free-viewing task. influences overt visual attention. In these experiments, two In a more structured visual search task, this congruity bias grapheme–color synesthetes viewed colored letters while their caused synesthetes to rapidly fixate and identify congruently eye movements were monitored. Letters were presented in colored target letters, but led to problems in identifying incon- colors that were either congruent or incongruent with the syn- gruently colored target letters. The results are discussed in terms esthetes’ colors. Eye tracking analysis showed that synesthetes of their implications for perception in synesthesia. & INTRODUCTION on overt eye movements aside from its already docu- Synesthesia is a fascinating condition in which ordinary mented influence on covert attention. First, it has been stimuli elicit unusual sensory experiences. For some shown that overt and covert attention can be dissoci- synesthetes, sounds can have a specific color (Ward, ated (e.g., Klein, 1980), and so demonstrations that syn- Huckstep, & Tsakanikos, 2006) or tastes can elicit spe- esthesia influences covert attention do not necessarily cific tactile and shape impressions (Cytowic, 1989, 1993). imply that synesthesia will also influence overt eye Synesthesia can even involve multiple extraordinary sen- movements. Second, our goal was to investigate spa- sory experiences for a single inducing stimulus (Dixon, tial shifts of attention that operate in everyday life and Smilek, & Merikle, 2004). An interesting and important these often involve fixating objects so that they can be characteristic of synesthesia is that the unusual sensory processed by the high acuity portion of the retina (i.e., associations synesthetes experience are not simply epi- the fovea; see Findlay & Gilchrist, 2003). Finally, dem- phenomenal, but rather appear to profoundly influence onstrating that synesthesia influences overt eye move- the synesthete’s cognition. Studies have shown synes- ments would also provide evidence that synesthesia can thesia can have a substantial impact on memory (Baron- have a direct impact on the oculomotor system. Cohen et al., 2007; Smilek, Dixon, Cudahy, & Merikle, To evaluate whether synesthesia influences eye move- 2002; Luria, 1968), emotional experiences (Callejas, ments, two individuals with grapheme–color synesthesia Acosta, & Lupia´n˜ez, 2007; Smilek, Malcolmson, et al., were studied. D. E. is a 20-year-old male undergraduate 2007; Ramachandran & Hubbard, 2001), perception (e.g., student at the University of Waterloo. For D. E., seeing Nikolic´, Lichti, & Singer, 2007; Palmeri, Blake, Marois, black letters and numbers elicits the highly specific color Flanery, & Whetsell, 2002; Ramachandran & Hubbard, experiences typical of grapheme–color synesthesia. For 2001; Smilek, Dixon, Cudahy, & Merikle, 2001), and co- example, D. E. describes the letter D as having an un- vert attention (Smilek, Callejas, Dixon, & Merikle, 2007; sightly shade of light pink, whereas the letter E and the Smilek, Dixon, & Merikle, 2003; Palmeri et al., 2002). number 3 are both a similar bright green. These color In the present article, we further explored the func- photisms are perceived as though emanating from, or tional impact of synesthesia by evaluating whether syn- overlaid on top of, the letters and numbers themselves, esthesia can influence overt exploratory behavior as and are present even when the graphemes are in his indexed by eye movements. There are several reasons peripheral vision. K. S. is a 23-year-old female under- for our interest in assessing the influence of synesthesia graduate student also at the University of Waterloo. Like D. E., when K. S. sees black letters and numbers she experiences highly specific colors that appear to be 1University of Waterloo, Ontario, Canada, 2Trent University, On- overlaid on top of the letters and numbers. Thus, K. S.’s tario, Canada and D. E.’s synesthetic experiences fit the definition D 2008 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 21:2, pp. 246–258 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn.2009.21048 by guest on 02 October 2021 of a ‘‘projector’’ synesthete as defined by Dixon et al. colors for those letters, and half were shown in colors (2004). Illustrating the variability one finds when compar- that were incongruent with the synesthete’s colors. We ing experiences across synesthetes (see Rich, Bradshaw, therefore anticipated that synesthetes would fixate con- & Mattingley, 2005; Simner et al., 2005; Calkins, 1893), gruently colored letters significantly more often and for unlike D. E., K. S. experiences a deep maroon color for a longer duration, on average, than incongruently col- the letter D and a dark green color for the letter E; as ored letters. well, she reports having to look directly at a grapheme in order to see its color. Nonetheless, for any given syn- esthete, the grapheme–color pairings reported are re- EXPERIMENT 1 markably invariant over time (Dixon, Smilek, Cudahy, & Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/21/2/246/1759957/jocn.2009.21048.pdf by guest on 18 May 2021 Merikle, 2000; Odgaard, Flowers, & Bradman, 1999; Two synesthetes first identified the colors they associate Baron-Cohen, Harrison, Goldstein, & Wyke, 1993; Svartdal with each letter of the English alphabet. The two syn- & Iversen, 1989) and both D. E. and K. S. report that for esthetes and two groups of nonsynesthetic participants as long as they can remember, they have consistently then viewed displays containing colored letters. Half of experienced the same color for a given letter every time the letters were congruently colored (for the synes- they saw it. thetes) and half were incongruently colored. The par- The purpose of Experiment 1 was to show that ticipants were required simply to view each display until grapheme–color synesthesia can influence overt atten- they felt they had seen all of the letters it contained. tion, as measured by eye movements. To demonstrate Eye movements were monitored throughout the free- this, we capitalized on the fact that people, in general, viewing task in order to evaluate the specific prediction prefer to look at things they like, and avoid looking that synesthetes would show a congruity bias in their at things they do not (Shimojo, Simion, Shimojo, & viewing behavior. To recap, for synesthetes, congruently Scheier, 2003). In order to understand how our prefer- colored letters should be fixated significantly more often ence to look at things we like and avoid things we do and for a longer duration, on average, than incongru- not would play out in grapheme–color synesthesia, how- ently colored letters. ever, one must first understand synesthetic–color con- gruity. For K. S., seeing a black E elicits the experience Methods of the black E along with a dark green overlay. Because text is most frequently presented as black characters on Participants a white background (books, newspapers, Microsoft A 20-year-old man with grapheme–color synesthesia Word documents, etc.), this combination is the standard (D. E.) participated in the approximately 1-hr free-viewing combination of physical color (black) and photism color task. As well, a 23-year-old woman with grapheme–color (green) that K. S. is used to experiencing. Synesthetes synesthesia (K. S.) participated in the same 1-hr free- report that. on those rare occasions when instead of a viewing session. Six nonsynesthetic students from the black letter the letter is presented in the ‘‘right’’ (i.e., University of Waterloo served as yoked controls for each congruent) color, this combination of physical color synesthete (i.e., they were presented with exactly the (green) and synesthetic color (green) looks and some- same letter displays in the same order, and were required how ‘‘feels’’ right. By contrast, if the letter is presented to perform the same task as the synesthete to whom they in the ‘‘wrong’’ (incongruent) color (e.g., red), synes- were yoked). Yoked control participants received partial thetes experience two conflicting colors: the ‘‘wrong’’ credit toward their Introductory Psychology course as color (red) in which the letter is physically presented, compensation. One of the nonsynesthete participants and the ‘‘right’’ color (green)—the color of their synes- yoked to D. E. was removed because of a disproportion- thetic photism induced by the letter. Synesthetes describe ately large number of fixations falling on entirely blank these incongruently colored graphemes as ‘‘jarring,’’ portions of the screen. This participant was replaced. ‘‘nausea inducing,’’ and ‘‘difficult to look at.’’ In short, synesthetes report they like looking at congruently col- Stimulus Displays ored stimuli and dislike looking at incongruently colored stimuli. If our propensity to spend more time looking Displays were created