Multi-Sensory Experience of Color with Poetry in Visual Arts Appreciation of Persons with Visual Impairment

electronics

Article ColorPoetry: Multi-Sensory Experience of Color with Poetry in Visual Arts Appreciation of Persons with Visual Impairment

Jun-Dong Cho * and Yong Lee

Department of Electronic and Electrical Engineering, Sungkyunkwan University, Suwon, Gyeonggi-do 16419, Korea; [email protected] * Correspondence: [email protected]; Tel.: +82-10-4332-7127

Abstract: Visually impaired visitors experience many limitations when visiting museum exhibits, such as a lack of cognitive and sensory access to exhibits or replicas. Contemporary art is evolving in the direction of appreciation beyond simply looking at works, and the development of various sensory technologies has had a great influence on culture and art. Thus, opportunities for people with visual impairments to appreciate visual artworks through various senses such as hearing, touch, and smell are expanding. However, it is uncommon to provide a multi-sensory interactive interface for color recognition, such as integrating patterns, sounds, temperature, and scents. This paper attempts to convey a color cognition to the visually impaired, taking advantage of multisensory coding color. In our previous works, musical melodies with different combinations of pitch, timbre, velocity, and tempo were used to distinguish vivid (i.e., saturated), light, and dark colors. However, it was rather difficult to distinguish among warm/cool/light/dark colors with using sound cues only. Therefore, in this paper, we aim to build a multisensory color-coding system with combining sound and poem such that poem leads to represent more color dimensions, such as including warm Citation: Cho, J.-D.; Lee, Y. and cool colors for red, orange, yellow, green, blue, and purple. To do this, we first performed an ColorPoetry: Multi-Sensory implicit association test to identify the most suitable poem among the candidate poems to represent Experience of Color with Poetry in colors in artwork by finding the common semantic directivity between the given candidate poem Visual Arts Appreciation of Persons with voice modulation and the artwork in terms of light/dark/warm/color dimensions. Finally, we with Visual Impairment. Electronics conducted a system usability test on the proposed color-coding system, confirming that poem will 2021, 10, 1064. https://doi.org/ 10.3390/electronics10091064 be an effective supplement for distinguishing between vivid, light, and dark colors with different color appearance dimensions, such as warm and cold colors. The user experience score of 15 college Academic Editors: Juan M. Corchado students was 75.1%, that was comparable with the color-music coding system that received a user and Rui Pedro Lopes experience rating of 74.1%. with proven usability.

Received: 2 March 2021 Keywords: visual impairment; accessibility; aesthetics; color; multi-sensory; museum exhibits Accepted: 25 April 2021 Published: 30 April 2021

Publisher’s Note: MDPI stays neutral 1. Introduction with regard to jurisdictional claims in Multimodal (or multisensory) integration refers to the neural integration or combi- published maps and institutional affil- nation of information from different sensory modalities (the classic five senses of vision, iations. hearing, touch, taste, and smell, and, perhaps less obviously, proprioception, kinesthesis, pain, and the vestibular senses), which gives rise to changes in behavior associated with the perception of and reaction to those stimuli. Information is typically integrated across sensory modalities when the sensory inputs share certain common features [1]. Copyright: © 2021 by the authors. Cross-modality refers to the interaction between two different sensory channels. Cross- Licensee MDPI, Basel, Switzerland. modal correspondence is defined as the surprising associations that people experience This article is an open access article between seemingly unrelated features, attributes, or dimensions of experience in different distributed under the terms and sensory modalities. conditions of the Creative Commons The intuitive strategies based on cross-modal analogy, association, and symbolism are Attribution (CC BY) license (https:// suitable for creating a design that provides connections between the senses, which directly creativecommons.org/licenses/by/ appear appropriate and easy to interpret [2]. 4.0/).

Electronics 2021, 10, 1064. https://doi.org/10.3390/electronics10091064 https://www.mdpi.com/journal/electronics Electronics 2021, 10, 1064 2 of 24

Although many studies have been conducted on the cross-sensation between the sight and other senses, there are not many studies on the cross-modality between the non-visual senses. Through the investigation of weak synesthesia that is perceived at the same time by intersecting various senses such as audio, touch, smell, etc., other sensory information can be connected to a specific form of color information. In this task, so far, efforts have been made to create different single-mode sensory perceptions for color, for example, color–sound, color–tactile pattern, color–odor, etc. In our previous studies, colors were expressed as embossed tactile patterns for recognition by finger touches, temperature, sound, and smell to provide a rich art experience to person with visual impairments. The Black Book of Colors by Cottin [3] described the experience of a fictional blind child named Thomas, who describes color through association with certain elements in his environment. This book highlights the fact that blind people can gain experience through multisensory interactions: “Thomas loves all colors because he can hear, touch, and taste them.” An accompanying audio explanation provides a complementary way to explore the overall color composition of an artwork. When tactile patterns are used for color transmission, the image can be comprehensively grasped by delivering graphic patterns, painted image patterns, and color patterns simultaneously [4]. This suggests to us the possibility and the justification for developing a new way of appreciating works in which the colors used in works are subjectively explored through the non-visual senses. In other words, it can be inferred that certain senses will be perceived as being correlated with certain colors and concepts through unconscious associations constructed with the concepts. However, while this helps visually impaired users perceive and understand colors in completely different ways, it becomes necessary to integrate all perceptual sensations into a single multi-sensory system, where all the senses are perfectly connected and interchanged. This concept is in line with the theory of emotional intelligence, which defends the organic functioning of people, as a whole, in interaction between the emotional, the sensory, and the cognitive [5,6]. According to Damásio [5], feelings are basically the mind’s interpretation of the state of the body, and in order for reason to be truly rational, it must be based on emotional cues from the body. Emotional memories are not easily erased from memory and remain the longest. Tacit memories that did not easily emerge above consciousness can suddenly come to mind through the opportunity of contact with any part of one’s body [5]. Audio description does provide a useful service for those with low or no visual acuity [7,8]. Schifferstein [9] observed that vivid images occur in all sensory modalities. The quality of some types of sensory images tends to be better (e.g., vision, audition) than of others (e.g., smell and taste) for sighted people. The quality of visual and auditory images did not differ significantly. Therefore, training these multi-sensory experiences introduced in this paper may lead to more vivid visual imageries or seeing with the mind’s eye. However, such descriptions are often monolithic accounts, missing the opportunity for alternative messages to be delivered and received. Making art accessible to the visually impaired requires the ability to convey explicit and implicit visual images through non- visual forms. It argues that a multi-sensory system is needed to successfully convey artistic images. It also designed a poetic text audio guideline that blends sounds and effects to translate the work into an ambiguous artistic sound text [9]. Audio or verbal description on an artwork generally attempts to explain the painting without expressing the individual subjectivity. When making visual art accessible to the visually impaired, it is not enough to describe the colors and situations portrayed because that objective information does not attach to anything in their experience. Using expressions that contain the sensibility of poems, color, and situation can be matched with the parts of each painting. Poem is a piece of writing that has features of both speech and song, whereas poetry is the art of creating these poems. Throughout the poem, seeing and hearing are used to understand color. Imagery is a literary device that refers to the use of figurative language Electronics 2021, 10, 1064 3 of 24

to evoke a sensory experience or create a picture with words for a reader. By utilizing effective descriptive language and ﬁgures of speech, writers appeal to a reader’s senses of sight, taste, smell, touch, and sound, as well as internal emotion and feelings. Poetic imagery provides sensory details to create clear and vivid descriptions. This appeals to a reader’s imagination and emotions as well as their senses [10]. The connection between color and poem has not been studied through existing research. In this paper, we explore how color can be explained by poetry so that visually impaired people feel or perceive color through poetry. However, the connection between color and poetry should be clearly matched. When appreciating color in artworks using poetry, we need to explore whether color concept and poem interfere with each other, causing confusion in color perception, or synergizing with each other to positively affect color perception. A system usability test is performed to evaluate whether it helps to easily recognize the color.

2. Background 2.1. Review on Ekphrasis In poetry, synesthesia refers specifically to figurative language that includes a mixing of senses. For example, saying “He wore a loud yellow shirt” is an example of synesthesia, as it mixes visual imagery (yellow) with auditory imagery (loud) [11]. Here is another example “The Loudness of Color” by Jennifer Betts [12]: The music of white dances softly around The soft silence and blue are bound Purple is calm, the sound soft and sweet The color lightness of a rainbow is a hypnotic beat In yellow, the silence is loud While red is a yell, robust and proud Red does not actually yell, but many would describe it as loud. Using sound to describe color makes it fun and interesting for the reader. The art of writing poetry about paintings is known as ekphrasis [13–15]. Ekphrasis’ technology was used to convey description, imagination, and emotion (vibrance) to direct visual artworks and played a role in enhancing the value of art. Edward Hirsch (1994) said, “works of art initiate and provoke other works of art” [16]. Color theory is associated with the Bauhaus artists Johannes Itten, Josef Albers, and Paul Klee. Itten wrote “The Art of Color”, The Elements of Color and Albers’ Interaction of Color, and Paul Klee notably produced picture poems, concerned with colors and words. The work of many contemporary poets has also been influenced by the visual arts: Rainer Maria Rilke, W.B. Yeats, W.H. Auden, and William Carlos Williams [17], Jane Flanders, Anne Sexton, X. J. Kennedy, and Gershon Hepner (Table1), are a few. The examples of ekphrases are shown in Table1. Koch [18] and Routman [19] made common observations on the practice of teaching poetry, specifically in regard to the importance of utilizing a range of quality poetry written by children and adults. Koch prefers to read poetry examples and then allow students to be influenced and imitate other poets. It makes them feel more open to understanding what others have written. Previous works mainly focus on generating poetry given keywords or other text information, while visual inspirations for poetry have been rarely explored. Electronics 2021, 10, x FOR PEER REVIEW

Electronics 2021, 10, x FOR PEER REVIEW Electronics 2021Electronics, 10, x FOR 2021 PEER, 10 REVIEW, x FOR PEER REVIEW

Electronics 2021, 10, 1064 Table 1. Examples of ekphrasis (a): artworks, (b): poems. 4 of 24 Table 1. Examples of ekphrasis (a): artworks, (b): poems. Table 1. ExamplesTable of1. ekphrasisExamples (ofa): ekphrasis artworks, ( a(b):): artworks, (poems.a) Artworks (b): poems. (Marcela) Artworks Duchamp (a) TableArtworks 1. Examples(a ) Artworks of ekphrasis (a): artworks, (b): poems. Nude Descending a MarcelStair- DuchampMark Rothko Marcel DuchampMarcel Duchamp (a) Artworks case No.2, Nude1912 DescendingNo. 13 (White, a Stair- Red on MarkYel- Rothko Vincent van Gogh VincentNude Descending van GoghNude Descending a Stair- a Stair-Mark RothkoMark Rothko Marcel Duchamp case No.2, 1912 No. 13 (White, Red on Yel- Arles, 1888 Vincent van GoghThe Starry Night, 1889Vincent vanPhiladelphia Gogh Museum of low), 1958 Vincent van Gogh case No.2, 1912case NoNude.2,No.Descending 1912 13 (White, a StaircaseNo. Red 13 No(White, on.2, Yel-1912 Red on Yel- Vincent vanVincent Gogh van Gogh Vincent van Gogh Art PhiladelphiaMuseum Museum of of Modern ArtMarklow), Rothko 1958 Van Gogh Museum Arles, 1888Modern Philadelphia MuseumPhiladelphiaThe Museumof ArtStarry ofNight, MuseumPhiladelphia 1889 oflow), Museum 1958 of Artlow), 1958 VincentArles, van 1888 Gogh Arles, 1888 The Starry Night,The VincentStarry 1889 van Night, Gogh 1889 Art No.Museum 13 (White, of Red Modern on Yellow), Art 1958 Arles, 1888 Van Gogh Museum Modern Museum of Art Museum of Modern Art Van Gogh MuseumVan Gogh MuseumModern MuseumModernThe of Starry MuseumArt Night, 1889 of Art Art Art Museum of MuseumModern Art of Modern Art Van Gogh Museum Modern Museum of Art

(b) Poems(b) Poems ( b) Poems Nude Descending a Stair- White, Red on YellowWhite, Red on Yellow Van Gogh’s Bed Van Gogh’s Bed The Starry(b) NightPoemsThe Starry(b) Poems Night Nude Descending a Staircase case Nude DescendingGershon a Stair- Hepner,White, Gershon Red Hepner,on Yellow Jane Flanders, 2015 [20] Jane Flanders, 2015 Van[20]Anne Gogh’s Sexton, Bed 1961AnneNude [21 Sexton,] Descending 1961Nude The[21] Descending a StarryStair- NightX.White, a J. Stair- Kennedy, Red 1961 White,on Yellow [22] Red on Yellow Van Gogh’sVan Bed Gogh’s Bed The Starry NightThe Starry Night X. J. Kennedy, 1961 [22] case 2008 [23] Gershon2008 Hepner, [23] Jane Flanders, 2015 [20] caseAnne Sexton,case 1961 [21]Gershon Hepner,Gershon Hepner, Jane Flanders,is orange,Jane 2015 Flanders, [20] 2015 [20]Anne Sexton,Anne The1961 townSexton, [21] does The 1961 not existtown [21] does not exist X. J. Kennedy, 1961 [22] 2008 [23] like Cinderella’s coach, like except where one black-hairedX. treeJ. Kennedy, slips X. J.1961 Kennedy, [22] 1961 [22]2008 [23] 2008 [23] ToeToe uponupon toe, toe, a snowing a snowing flesh, the sun when he looked it up like a drowned womanexcept intowhere the hot one sky. black-hairedThe town does not exist The town doesThe not town exist does not exist A gold of lemon, rootToe and upon rind, toe, a snowing straight in the eye. The town is silent. The night boils with eleven stars. flesh, is orange, Toetree upon slipsexcept toe,Toe a where uponsnowing toe,oneShe ablack-haired sifts snowing in sunlight down the stairs is narrow, he sleeps alone, except whereexcept oneOh black-haired starrywhere night! one Thisblack-haired is how A gold of lemon, root and flesh, White and red and yellow Rothko thought more flesh,With nothing on. Nor on her mind. tossing between two pillows,like Cinderella’sI wantcoach, to die.likeis Itorange, moves. up like They a aredrowned all alive.flesh, woman treeinto slips mellow is orange, is orange, tree slips tree slips We spy beneathrind, the A banistergold of lemon, root and while it carried him Even the moon bulges in itsA orange gold ironsofup lemon, Alike gold a rootdrowned of lemon,and woman root and into than mountains that are grandiose, the sun when helike looked Cinderella’s it coach, likethe hot sky. A constant thresh of thigh on thigh– like Cinderella’sbumpilylike to theCinderella’s coach, ball. like coach,up like like a drowned toup push like children, womana drowned like into a god, woman from itsinto eye. She sifts in sunlight down rind, or portraits. Figures can be gross rind, the hotrind,Her sky. lips imprint the swinging air is clumsy, straight inThe thethe old eye. unseensun when serpent heThe swallows looked town up itis thesilent. stars. The night White and red andand mountains yellow overwhelm the view, the sun whenthe he sun looked when it he looked it the hot sky. the hot sky. That partsthe to let stairs herShe parts sifts go by. in sunlight down but friendly. A peasant straightOh starry in night! the Thisboils eye.She is with how sifts eleven inThe sunlightShe townstars. sifts isdownin silent. sunlight The down night Rothko thoughtbut Whitemore colors, mel- whenand red they’re and few yellow and pure, The town is silent. The night One-woman waterfall,White she and wears thered stairs and yellow builtstraight the frame; in and thestraight old eye. wife beat in the eye.The town is silent. TheI want night to die: WithWhite nothing and red on. and Nor yellow on achieve a balance of pure reason is narrow, he sleeps alone, Oh starry night!the This stairsboils is howwiththe eleven stairsHer slow stars. descent like a long cape low Rothko thought more mel- the mattress till it rose like meringue. boils withinto elevenboils that rushing withstars. eleven beast of stars. the night, Rothko thoughtRothkoWith more thought nothing mel- moreon. Nor mel- on in every place, in every season. And pausing,her mind. on the final stair is empty, tossing between twosuckedis narrow, pillows, up by he that sleeps greatI want dragon, alone, Withto die. to nothing split It moves.OhWith starry on. They nothingNor night! on on.This Nor is how on than mountains that are low is narrow, heis sleepsnarrow, alone, he sleeps Oh alone, starry night!Oh starry This isnight! how This is how WeCollects spy her beneath motionslow the into banis- shape.herlow mind. morning light pours in while it carriedtossing himfrom between my life two with pillows, no flag,are all her alive.I want mind. to die.her Itmind. moves. They grandiose,than mountains that are tossinglike wine, between melody,tossing two fragrance, between pillows, two I pillows, want to die. I want It moves. to die.no They belly,It moves. They than mountainster thanWe thatspymountains beneathare that the arebanis- bumpily to the ball.while it carriedEven himtheWe moon spy beneathbulgesWe inspy the itsare beneathbanis- or- all alive. the banis- or portraits. Figures cangrandiose, thewhile memory it carried of happiness.while him it carried him are all alive.are noall cry. alive. A constantgrandiose, thresh of thigh grandiose, ter bumpily to the ball. ange ironsEventer the moonter bulges in its or- be grossor portraits. Figures can bumpily to thebumpily ball. to theEven ball. the moonEven bulges the moon in its bulges or- in its or- or portraits.on thigh--or FiguresA portraits. constant can threshFigures of can thigh is clumsy, to pushA constantchildren, threshAlike constant a god,ofange thigh thresh irons of thigh and mountains over- be gross ange irons ange irons Her lips imprintbe gross the swing- beon gross thigh-- but friendly. A peasantis clumsy, from itson toeye. thigh-- push children,on thigh-- like a god, whelm the view,and mountains over- is clumsy, is clumsy, to push children,to push like children, a god, like a god, and mountainsing airHer and lips over-mountains imprint theover- swing- built the frame; andbut old friendly. wife TheA peasant oldHer unseen lips imprint serpentHer lips the swal-from imprintswing- its eye. the swing- but colors, when they’rewhelm the view, but friendly.but A peasantfriendly. A peasant from its eye.from its eye. That partswhelm to let the her view,whelm parts ingthe airview, beatbuilt the frame; and oldlows wife up theTheing stars. airold unseening airserpent swal- few and pure,but colors, when they’re built the frame;built and the oldframe; wife and Theold oldwife unseen The serpentold unseen swal- serpent swal- but colors,go by. whenbut That colors, they’re parts when to let they’re her parts the mattress till it rose like me-beatOh starryThat night! parts ThistoThat let lowsis herparts how partsup to the let stars. her parts achieve a balance of purefew and pure, beat beat lows up the lowsstars. up the stars. One-womanfew andwaterfall, pure,few she andgo pure,by. ringue.the mattress till it rose likeI want me- togoOh die: by. starry night!go by. This is how reason achieve a balance of pure the mattressthe till mattress it rose like till me- it roseOh like starry me- night!Oh starry This isnight! how This is how achievewears a balanceachieveOne-woman of a pure balance waterfall, of pure she ringue.into thatOne-woman rushing beastOne-woman waterfall, ofI want the she waterfall,to die: she in every place, in every reason ringue. ringue. I want to die:I want to die: Her slow descentreason like areason wears is empty, night,intowears that rushingwears beast of the season. in every place, in every into that rushinginto that beast rushing of the beast of the in everylong place, capein Her every in everyslow place, descent in every like a morning light pours inis empty,sucked Her up slow by that descentHer great slow like night, descent a like a season. is empty, is empty, night, night, And pausing,season. on the finalseason.long cape like wine, melody, fragrance,morning light poursdragon, in long to splitsucked cape longup by cape that great morning lightmorning pours inlight pourssucked in up bysucked that great up by that great stair And pausing, on the final the memory oflike happiness. wine, melody, from fragrance,And my life pausing, withAnd no on dragon,flag, pausing,the final to on split the final like wine, melody,like wine, fragrance, melody, fragrance,dragon, to splitdragon, to split Collects her motions into stair the memory of happiness.no belly,stairfrom my lifestair with no flag, the memorythe of happiness.memory of happiness.from my lifefrom with my no lifeflag, with no flag, shape. Collects her motions into Collectsno cry. her Collects motionsno her into belly, motions into no belly, no belly, shape. shape. noshape. cry. no cry. no cry. Good poems produce sensational images in compressed and refined language. When you read a poem, theGood scene poems it describes produce seems sensational to unfold images before in your compressed eyes. Association and refined is a language. When Good poemsGood produce poems sensational produce imagessensational in compressed images in compressed and refined andlanguage. refined When language. When you read a poem, the scene it describes seems to unfold before your eyes. Association is a you read a poem,you read the ascene poem, it describesthe scene itseems describes to unfold seems before to unfold your beforeeyes. Association your eyes. Associationis a is a

Electronics 2021, 10, 1064 5 of 24

Good poems produce sensational images in compressed and reﬁned language. When you read a poem, the scene it describes seems to unfold before your eyes. Association is a phenomenon in which one idea evokes another. Five images found in poetry, for example, are as follows. 1. Visual: the light blue sky 2. Sound: the wind attenuating the acoustic image 3. Olfactory: The smell of grass ﬂowers 4. Taste: sweet chocolate 5. Tactile: warm stars 6. Synesthesia: blue whistling sound

2.2. Review on Review on Color Association with Sound Recently, the following works were devised to materialize multi-sensory appreciation for the color of artworks for the visually impaired [24]. An experiment comparing the cognitive capacity for color codes found that users could intuitively recognize 24 chromatic and 5 achromatic colors with tactile pictogram codes [25] and 18 chromatic and 5 achromatic colors with sound codes [26]. Cho et al. [25] presented a tactile color pictogram system to communicate the color information of visual artworks. The tactile color pictogram uses the shape of sky, earth, and human derived from the oriental philosophy of heaven, earth, and human as a metaphor. The tactile color pictogram used a slightly larger cell size compared to most tactile patterns currently used to indicate color, but code for more colors due to its simple structure. With user experience and identification tests, conducted with 23 visually impaired adults, the effectiveness of the tactile color pictogram suggested that they were helpful for the participants. Colors can thus be recognized easily and intuitively by touching the different patterns. What the art teacher wanted to do most with her blind students was to have them imagine colors using a variety of senses—touch, scent, music, poetry, or literature. A comprehensive survey of associations between color and sound can be found in [24], including how different color properties such as value and hue are mapped onto acoustic properties such as pitch and loudness. Cho et al. [26] developed a sound coding color (Table2) to express red, orange, yellow, green, blue, and purple using a melody (length: about 10 s) excerpted from a Classic music video with different musical instruments, intensity, and pitch of sound to express vivid, light, and dark colors. In Reference [26], musical instruments were classified for each color to ensure that they would be easily distinguished from one another. Red, a representative warm color, is expressed by a violin that plays a passionate and strong melody. A trumpet plays a high-pitched melody with energy, as if a bright light were expanding, to simulate yellow bursts. Orange is expressed by a viola playing a warm yet energetic melody. Green, which makes the eyes feel comfortable and psychologically stable, is expressed by a fresh oboe that plays a soft melody. Blue, a representative cold color, is expressed by a cello that plays a low, calm melody. Violet, where warm red and cold blue coexist, is expressed by a pipe organ that plays a magnificent yet solemn melody. Vivid colors and bright and dark colors were distinguished through a combination of pitch, instrument tone, intensity, and tempo. High color lightness used a small, light, particle-like melody and high-pitch sounds, and a bright feeling was emphasized by using a melody of relatively fast and high notes. For low color lightness, a slow, dull melody with a relatively low range was used to create a sense of separation and movement away from the user. Electronics 2021,, 10,, xx FORFOR PEERPEER REVIEWREVIEW ElectronicsElectronicsElectronics 2021 20212021,, ,10, 1010,, ,xx, x xFORFOR FORFOR PEERPEER PEERPEER REVIEWREVIEW REVIEWREVIEW Electronics Electronics 2021 2021,, 10, 10,, xx, xFORFOR FOR PEERPEER PEER REVIEWREVIEW REVIEW Electronics ElectronicsElectronics 2021 20212021, ,,10 1010, ,,x xx FOR FORFOR PEER PEERPEER REVIEW REVIEWREVIEW ElectronicsElectronics 2021 2021, 10, 10, x, xFOR FOR PEER PEER REVIEW REVIEW Electronics 2021,, 1010,, 1064x FOR PEER REVIEW 6 of 24 Electronics Electronics 2021 2021,, 10, 10,, xx, xFORFOR FOR PEERPEER PEER REVIEWREVIEW REVIEW

Table 2. Sound coding color [26] (the sound files developed in this research are provided sepa- TableTableTable 2. 2.2. Sound SoundSound coding codingcoding color colorcolor [26] [26][26] (the (the(the sound soundsound files filesfiles developed developeddeveloped in inin this thisthis research researchresearch are areare provided providedprovided sepa- sepa-sepa- ratelyTableTable as 2. 2. aSound Soundsupplement coding coding material).color color [26] [26] (the (the sound sound files files developed developed in in this this research research are are provided provided sepa- sepa- ratelyTableTableTablerately as2. 2.2.as Sound a SoundSound asupplement supplement coding codingcoding material).color color colormaterial). [26] [26][26] (the (the (the sound soundsound files filesfiles developed developeddeveloped in inin this thisthis research researchresearch are areare provided providedprovided sepa- sepa-sepa- TableratelyratelyTablerately 2. as as 2.as Sound aaSound a supplementsupplement supplement coding coding color material). material). colormaterial). [26] [26] (the (the sound sound files files developed developed in in this this research research are are provided provided sepa- sepa- TableratelyratelyratelyTable 2.as 2. as as2. SoundSounda aaSound supplement supplementsupplement codingcoding coding colormaterial). color material). material).color [[26]26 [26]] (the(the (the soundsound sound ﬁlesfiles files developeddeveloped developed inin in thisthis this researchresearch research are are are provided provided provided separately sepa- sepa- ratelyrately asInstrumentInstrument asInstrument a asupplement supplement andand and material). material). MusicMusic Music LLL VVV DDD ratelyasrately a Supplemental asInstrumentInstrument InstrumentasInstrument a asupplement supplement Materials). and andand and material). material). Music MusicMusic Music LLL L VVVV DDDD R-Vn.InstrumentInstrumentInstrument High-frequency and andand Music MusicMusic banded LLL VVV DDD R-Vn.R-Vn.R-Vn.InstrumentInstrument High-frequency High-frequencyHigh-frequency and and Music Music banded bandedbanded LL VV DD R-Vn.R-Vn.InstrumentInstrumentInstrument stringHigh-frequency High-frequency instrument and andand Music Music banded banded LL V V D DD R-Vn.R-Vn.R-Vn.Instrument High-frequencystring High-frequencyHigh-frequencystring instrument instrument and Music banded bandedbanded L V D R-Vn.R-Vn. High-frequencystring stringHigh-frequencystring instrumentinstrument instrument banded banded 1. R-L.wav 1. R-S.wav 1. R-D.wav Tchaikovsky:R-Vn.Tchaikovsky:R-Vn. High-frequencystring stringstringHigh-frequency Violin instrument instrument instrumentViolin Concerto Concerto banded banded in in 1.1.1. R-L.wav R-L.wavR-L.wav 1.1.1. R-S.wav R-S.wavR-S.wav 1.1.1. R-D.wav R-D.wavR-D.wav Tchaikovsky:R-Vn.Tchaikovsky:R-Vn. High-frequency stringHigh-frequencystring instrumentViolin Violininstrument bandedConcerto Concerto banded string in in 1.1. R-L.wav R-L.wav 1.1. R-S.wav R-S.wav 1.1. R-D.wav R-D.wav Tchaikovsky:Tchaikovsky: Violin DViolin Concerto Concerto in in 1.1. R-L.wavR-L.wav 1.1. R-S.wavR-S.wav 1.1. R-D.wavR-D.wav Tchaikovsky:Tchaikovsky:Tchaikovsky:stringstringinstrument instrument Violin ViolinViolininstrumentDD Concerto ConcertoConcerto in inin 1. R-L.wav 1. R-S.wav 1. R-D.wav Tchaikovsky:Tchaikovsky: Violin ViolinDDDD Concerto Concerto in in 1.1. R-L.wav R-L.wav 1.1. R-S.wav R-S.wav 1.1. R-D.wav R-D.wav Tchaikovsky: ViolinD Concerto in D 1.1. R-L.wav R-L.wav 1.1. R-S.wav R-S.wav 1.1. R-D.wav R-D.wav Tchaikovsky:Tchaikovsky: Violin ViolinDD Concerto Concerto in in O-Va.O-Va.O-Va. Medium-frequencyMedium-frequency Medium-frequencyDD O-Va.O-Va.O-Va.O-Va. Medium-frequency Medium-frequencyMedium-frequency Medium-frequencyDD bandedO-Va.bandedO-Va.O-Va.banded Medium-frequency Medium-frequencyMedium-frequency stringstring string instrumentinstrument instrument 2.O-L.wav 2.O-S.wav 2.O-D.wav O-Va.O-Va.bandedbandedO-Va. Medium-frequency Medium-frequency Medium-frequency string string instrument instrument banded 2.O-L.wav2.O-L.wav 2.O-S.wav2.O-S.wav 2.O-D.wav2.O-D.wav Stamitz:bandedbanded Viola string string Concerto instrument instrument in D 2.O-L.wav2.O-L.wav2.O-L.wav 2.O-S.wav2.O-S.wav2.O-S.wav 2.O-D.wav2.O-D.wav2.O-D.wav Stamitz:Stamitz:Stamitz:O-Va.bandedbandedbandedO-Va.string Medium-frequency Viola Medium-frequency Viola stringViola stringstring instrument Concerto Concerto Concerto instrument instrumentinstrument in inin D D D 2.O-L.wav2.O-L.wav 2.O-S.wav2.O-S.wav 2.O-D.wav2.O-D.wav Stamitz:Stamitz:bandedbanded Viola string Violastring Concerto instrumentConcerto instrument in in D D 2.O-L.wav 2.O-S.wav 2.O-D.wav Stamitz:Stamitz:Stamitz:bandedStamitz:banded Viola ViolaViolastring string Concerto ConcertoConcertoConcerto instrument instrument in in inin D D DD 2.O-L.wav2.O-L.wav 2.O-S.wav2.O-S.wav 2.O-D.wav2.O-D.wav Stamitz:Stamitz:banded Viola Violastring Concerto Concertoinstrument in in D D 2.O-L.wav2.O-L.wav 2.O-S.wav2.O-S.wav 2.O-D.wav2.O-D.wav Y-Trp. Brass instrument Stamitz:Stamitz:Y-Trp.Y-Trp.Y-Trp. Viola Brass ViolaBrassBrass Concerto instrumentConcerto instrumentinstrument in in D D Haydn:Y-Trp.Y-Trp. Trumpet Brass Brass instrumentConcerto instrument in E Haydn:Haydn:Y-Trp.Y-Trp.Y-Trp.Y-Trp. Trumpet Trumpet Brass Brass Brass instrument instrumentinstrumentConcerto Concerto in in E E 3.Y-L.wav 3.Y-S.wav 3.Y.-D.wav Haydn:Haydn:Haydn:Y-Trp.Y-Trp. TrumpetTrumpet Trumpet Brass Brass instrument instrument ConcertoConcerto Concerto inin in EE E 3.Y-L.wav3.Y-L.wav3.Y-L.wav 3.Y-S.wav3.Y-S.wav3.Y-S.wav 3.Y.-D.wav3.Y.-D.wav3.Y.-D.wav Haydn:Haydn: Trumpet Trumpetflatflat Concerto Concerto in E in ﬂat E 3.Y-L.wav3.Y-L.wav 3.Y-S.wav3.Y-S.wav 3.Y.-D.wav3.Y.-D.wav Haydn:Haydn:Y-Trp.Y-Trp. Trumpet Trumpet Brass Brassflatflat instrument Concertoinstrument Concerto in in E E 3.Y-L.wav3.Y-L.wav3.Y-L.wav 3.Y-S.wav3.Y-S.wav3.Y-S.wav 3.Y.-D.wav3.Y.-D.wav3.Y.-D.wav Haydn:Haydn: Trumpet Trumpetflatflatflat Concerto Concerto in in E E 3.Y-L.wav3.Y-L.wav 3.Y-S.wav3.Y-S.wav 3.Y.-D.wav3.Y.-D.wav Haydn:Haydn: Trumpet Trumpetflatflatflat Concerto Concerto in in E E flatflat 3.Y-L.wav3.Y-L.wav 3.Y-S.wav3.Y-S.wav 3.Y.-D.wav3.Y.-D.wav G-Ob.G-Ob. Woodwind Woodwindflatflat instrument instrument G-Ob.G-Ob.G-Ob. Woodwind Woodwind Woodwindflat instrument instrument instrument with G-Ob.G-Ob. Woodwind Woodwindwith reed instrument instrument G-Ob.G-Ob.G-Ob. Woodwind WoodwindWoodwindwithwithreed reed reed instrument instrumentinstrument 4. G-L.wav 4. G-S.wav 4. G-D.wav G-Ob.G-Ob. Woodwind Woodwindwithwithwith reedreed reed instrument instrument 4.4.4. G-L.wav G-L.wavG-L.wav 4.4.4. G-S.wav G-S.wavG-S.wav 4.4.4. G-D.wav G-D.wavG-D.wav Mozart:Mozart: Oboe Oboe Concerto Concerto inin C C 4.4. G-L.wav G-L.wav 4.4. G-S.wav G-S.wav 4.4. G-D.wav G-D.wav G-Ob.Mozart:G-Ob.Mozart:Mozart: Woodwind Woodwind Oboe OboewithOboewithwith Concerto reed reedConcertoreedConcerto instrument instrument in inin C CC 4.4. G-L.wavG-L.wav 4.4. G-S.wavG-S.wav 4.4. G-D.wavG-D.wav Mozart:Mozart: Oboewith Oboewith reedConcerto reedConcerto in in C C 4. G-L.wav 4. G-S.wav 4. G-D.wav Mozart:Mozart:Mozart: Oboe OboeOboewithwith Concertoreed Concerto Concertoreed in inin C CC 4.4. G-L.wav G-L.wav 4.4. G-S.wav G-S.wav 4.4. G-D.wav G-D.wav Mozart:Mozart: Oboe withOboe Concertoreed Concerto in in C C 4.4. G-L.wav G-L.wav 4.4. G-S.wav G-S.wav 4.4. G-D.wav G-D.wav B-Vc. Low-frequency banded B-Vc.B-Vc.Mozart:B-Vc.B-Vc.Mozart: Low-frequency Low-frequency Low-frequencyLow-frequency Oboe Oboe Concerto Concerto banded banded bandedbanded in in string C C B-Vc.B-Vc. stringLow-frequency Low-frequencyinstrument instrument banded banded B-Vc.B-Vc.B-Vc. stringLow-frequency Low-frequencyLow-frequencystring instrument instrument banded bandedbanded 5.B-L.wav 5.B-S.wav 5.B-D.wav B-Vc.B-Vc.Bach: Low-frequencystring stringLow-frequencystring Cello instrumentinstrument Suiteinstrument No. banded 1banded in G 5.B-L.wav5.B-L.wav5.B-L.wav 5.B-S.wav5.B-S.wav5.B-S.wav 5.B-D.wav5.B-D.wav5.B-D.wav Bach:Bach: stringCello Cello Suiteinstrument Suite No. No. 1 1 in in G G 5.B-L.wav5.B-L.wav 5.B-S.wav5.B-S.wav 5.B-D.wav5.B-D.wav B-Vc.Bach:B-Vc.Bach: Low-frequencystring stringCello Low-frequencyCello instrument Suite instrumentSuite No. No. banded 1 banded1 in in G G 5.B-L.wav5.B-L.wav5.B-L.wav 5.B-S.wav5.B-S.wav5.B-S.wav 5.B-D.wav5.B-D.wav5.B-D.wav Bach:Bach:string stringCello Cello instrument Suiteinstrument Suite No. No. 1 1 in in G G 5.B-L.wav5.B-L.wav 5.B-S.wav5.B-S.wav 5.B-D.wav5.B-D.wav Bach:Bach:Bach:string Cello stringCelloCello instrument Suite SuiteinstrumentSuite No. No.No. 1 11 in inin G GG 5.B-L.wav 5.B-S.wav 5.B-D.wav P-Org.Bach:P-Org.Bach: Cello KeyboardCello Keyboard Suite Suite instrumentNo. instrumentNo. 1 1 in in G G 5.B-L.wav5.B-L.wav 5.B-S.wav5.B-S.wav 5.B-D.wav5.B-D.wav P-Org.P-Org.Bach:P-Org.Bach: KeyboardCello KeyboardCelloKeyboard Suite Suite instrument instrument No.instrument No. 1 1in in withG G withP-Org.Bach:P-Org. simultaneous Cello Keyboard Keyboard Suite instrumentexpressionsNo. instrument 1 in G withP-Org.withP-Org.P-Org.simultaneous simultaneous simultaneous Keyboard KeyboardKeyboard expressions instrument expressionsinstrumentinstrument expressions withwithP-Org.withP-Org. simultaneoussimultaneous simultaneous Keyboard Keyboard instrument expressionsinstrumentexpressions expressions 6. P-L.wav 6. P-S.wav 6. P-D.wav withwithwithMozart:P-Org.Mozart:P-Org.Mozart: simultaneous simultaneoussimultaneous Keyboard EineKeyboard Eine Eine Kleine Kleine Kleine instrument expressions Nachtmusikexpressionsinstrumentexpressions Nacht- Nacht- 6.6.6. P-L.wav P-L.wavP-L.wav 6.6.6. P-S.wav P-S.wavP-S.wav 6.6.6. P-D.wav P-D.wavP-D.wav withwithP-Org.Mozart:Mozart: simultaneous simultaneous Keyboard Eine Eine Kleine Kleine instrumentexpressions expressions Nacht- Nacht- 6.6. P-L.wav P-L.wav 6.6. P-S.wav P-S.wav 6.6. P-D.wav P-D.wav Mozart:Mozart: Eine Einemusik Kleine Kleine Nacht- Nacht- 6.6. P-L.wavP-L.wav 6.6. P-S.wavP-S.wav 6.6. P-D.wavP-D.wav withwithMozart:Mozart:Mozart: simultaneous simultaneous Eine EineEinemusikmusik Kleine KleineKleine expressions expressions Nacht- Nacht-Nacht- 6. P-L.wav 6. P-S.wav 6. P-D.wav Mozart:Mozart: Eine Einemusikmusikmusikmusik Kleine Kleine Nacht- Nacht- 6.6. P-L.wav P-L.wav 6.6. P-S.wav P-S.wav 6.6. P-D.wav P-D.wav Mozart:Mozart: Eine Einemusikmusikmusik Kleine Kleine Nacht- Nacht- 6.6. P-L.wav P-L.wav 6.6. P-S.wav P-S.wav 6.6. P-D.wav P-D.wav Mozart: Einemusikmusik Kleine Nacht- InIn Reference [[26],26], thethe overall overall color color composition composition of of Van Van Gogh’s Gogh’s “The “The Starry Starry Night” Night” was InInIn Reference ReferenceReferencemusikmusik [26], [26],[26], the thethe overall overalloverall color colorcolor composition compositioncomposition of ofof Van VanVan Gogh’s Gogh’sGogh’s “The “The“The Starry StarryStarry Night” Night”Night” wasexpressed expressedInInIn ReferenceReference Reference as a singleas a[26],[26], [26],single piece thethe the pieceoveralloverallof overall music of colorcolor musiccolor that compositioncomposition accounted compositionthat accounted for ofof ofcolor VanVan Vanfor Gogh’susingGogh’s colorGogh’s theusing “The“The “The tone, theStarryStarry Starry key, tone, tempo,Night”Night” Night” key, waswaswas Inexpressed InexpressedInexpressed Reference ReferenceReference as asas [26], a[26], [26],aa single singlesingle the thethe overall piece overalloverallpiecepiece of of of color colorcolor music musicmusic composition compositioncomposition that thatthat accounted accountedaccounted of ofof Van VanVan for forfor Gogh’s Gogh’sGogh’s color colorcolor using using“The using“The“The theStarry StarrytheStarrythe tone, tone,tone, Night” Night”Night” key, key,key, tempo,andwaswas pitchInexpressed Inexpressed andReference Reference of pitch the as instruments.as of [26], a [26], a thesingle single the instruments.the overall piece overallpiece To express of ofcolor colormusic Tomusic thecompositionexpr composition that highlythatess accounted theaccounted saturated highly of of Van Van for saturatedfor (vivid)Gogh’s Gogh’scolor color blue using “The using(vivid)“The of Starrythe theStarrythe blue tone, nighttone, Night” ofNight” key, sky,thekey, tempo,waswastempo,wastempo, In expressedexpressedexpressedIn Referenceand andandReference pitch pitchpitch asasas of [26], aofofa[26],a singlethe single singlethethe the theinstruments. instruments.instruments. overall piece piecepieceoverall ofof ofcolor color musicmusicmusic To ToTo composition expr exprcomposition expr thatthatthatessessess accountedaccounted accountedthe thethe highly ofhighly highlyof Van Van forfor forsaturated saturatedsaturatedGogh’s colorGogh’scolorcolor using usingusing“The (vivid) “The (vivid)(vivid) theStarry thetheStarry blue blue tone,bluetone,tone, Night” of Night” of of key, key,key,the thethe nightwaswhichtempo,tempo,wastempo, expressed sky,expressed dominates andand and which pitchpitch pitch as as dominates theofofa ofa single thethe single overallthe instruments.instruments. instruments. piece piecethe hue overallof ofof music themusic ToTo To hue picture, exprexpr exprthat thatofess esstheaccounted aaccounted the strong, thepicture, highly highly clear for a forsaturated strong, saturatedcolor melodycolor using clearusing (vivid) in(vivid) themelodythe the mid-rangeblue tone, bluetone, inof ofkey, thekey,the the nightwastempo,tempo,nighttempo,nightwas expressed sky, expressed sky, sky, and andand which which which pitch pitchpitch as as dominates of dominatesofadominatesof a singlethe thethesingle instruments. instruments.instruments. piece piecethe thethe overall ofoverall overallof music music To ToTo hue hueexpr hueexprexpr that thatof ofessofessess theaccounted thethe accounted the thethe picture, picture,picture, highly highlyhighly for afor saturateda asaturatedsaturatedstrong, strong,colorstrong, color using clear usingclear clear(vivid) (vivid)(vivid) melody the melodymelodythe blue bluebluetone, tone, ofin ofinof inkey, the key, thethethe mid-rangetempo,wasnighttempo,night excerpted sky, sky,and and waswhich whichpitch pitch excerpted from dominatesof ofdominates the thethe instruments. Bach instruments.from the the unaccompaniedthe overall overall Bach To To unacco hue expr hueexpr of essofessmpanied cellothethe thethe the picture,picture, picture, suitehighly highly cello No. aasaturated a saturatedsuitestrong,strong, strong, 1 to No. form clearclear (vivid) clear1(vivid) to the melodymelodyform melody baseblue blue the ofin in ofbasein thethe the thethe mid-rangetempo,nightnightmid-rangenightmid-rangetempo, sky, sky,sky, and and which was which whichwaswaspitch pitch excerpted excerptedexcerpted dominates ofdominatesdominates of the the instruments. instruments.from fromfrom the thethe the thethe overall overalloverall Bach BachBach To To unaccohue hueunaccohueexprunacco expr of ofofess essmpanied the mpaniedthempaniedthe the the picture, picture,picture, highly highly cello cellocello a asaturateda saturatedstrong, suite strong,strong, suitesuite No. No.No. clear clearclear(vivid) 1 (vivid) 11 to to to melody melodyformmelody formform blue blue the thethe ofin inin ofbase base basethe thethethe ofnightwholemid-rangenightmid-range the sky,whole song,sky, which waswhich wassong, and excerpted excerpted itdominates dominatesand is played it is from fromplayed the repeatedlythe the theoverall overall repeatedBach Bach hueunacco withouthueunaccolyly of of withoutwithout mpaniedthe thempanied interruption. picture, picture, interruption.interruption. cello cello a a strong, suitestrong, suite To expressNo. ToTo No.clear clear express1express 1to tomelody the formmelody formtwinkling thethe the the intwin- twin-in base thebase the nightmid-rangeofmid-rangemid-rangeofnight the the sky, whole sky,whole whichwas waswaswhich song, song, excerpted excerptedexcerpted dominates dominatesand and it it isfrom fromfromis played theplayed the the the theoverall overall Bach repeatedBachBach repeated unacco hue unacco unaccohuely ofly of without mpaniedthewithoutmpanied mpaniedthe picture, picture, interruption. interruption.cello cellocello a a strong,suite suitesuitestrong, No. No.No. To clearTo clear 1 express11 expressto toto melody form melodyformform the the thethe in twin- inbase twin- base basethe the mid-rangeofbrightofofmid-rangeof the thethe the whole yellowwholewhole whole was was song, song, ofsong,song, excerpted theexcerpted and andand stars,and it itit it aisfromisis fromis light played playedplayed played the particle-likethe Bach repeated repeatedBachrepeated repeated unacco unaccolyly melodylylyly without withoutwithoutmpaniedwithout withoutmpanied was interruption. interruption.interruption.interruption. interruption.cello extracted cello suite suite from No. No. To ToToTo To 1 Haydn’sexpress expressexpress1 expresstoexpress to form form the thethe Trumpetthe thethe the twin- twin- twin-twin- basetwin- base klingofklingofofkling the thethe bright bright whole wholebrightwhole yellow yellow song, song,yellowsong, and andofandof of thethe it theitit is isstars,isstars, stars, played playedplayed aa a lightlight repeatedlight repeatedrepeated particle-likeparticle-like particle-likelylyly without withoutwithout melodymelody melody interruption. interruption.interruption. waswas was extractedextracted extracted To ToTo express expressexpress fromfrom from the Haydn’stheHaydn’sthe Haydn’s twin- twin-twin- ofklingmid-rangeConcertoklingofmid-range the the brightwholebright whole and was was yellowsong,yellow playedsong, excerpted excerpted and of andof as the theit ait strong,isfrom stars,isfrom stars,played played the athe cleara light Bachlightrepeated Bachrepeated melody particle-like particle-likeunacco unaccolyly inwithout mpaniedwithoutmpanied the melody mid-range.melody interruption. interruption. cello cello was was suite suite The extracted extracted No. paintingNo.To To 1express 1expressto to from formfrom form was the Haydn’stheHaydn’s divided the twin- basetwin- base TrumpetklingTrumpetkling bright bright Concerto Concerto yellow yellow and ofand of the playedthe played stars, stars, as aas aalight alightstrong, strong, particle-like particle-like cl clearear melody melody melody melody in in thewas thewas mid-range. extractedmid-range. extracted from Thefrom The paintingHaydn’s paintingHaydn’s TrumpetofklingintoklingTrumpetklingof the the four bright brightwholebright whole Concerto lines,Concerto yellow yellowsong,yellow song, and and and of workedandofof the the theitplayed playedit is stars, isstars, stars,played withplayed as asa aa 16 light a lightlightarepeated strong, repeated barsstrong, particle-like particle-likeparticle-like per cl lylycllyear line, earwithoutwithout without melody melody producing melody melodymelody interruption.interruption. interruption. in in wasthe wasthewas a totalmid-range. extractedmid-range. extractedextracted ToTo of To 68expressexpress express bars from fromfromThe The played thetheHaydn’s paintingHaydn’stheHaydn’spainting twin-twin- twin- in wasklingTrumpetklingTrumpet divided bright bright Concerto Concerto yellowinto yellow four andof andof lines, the theplayed played stars, stars,and asworked a asa light a light astrong, strong, particle-like withparticle-like cl cl16earear bars melody melody melody permelody line, in in wasthe wasproducingthe mid-range.extracted mid-range.extracted a totalfrom fromThe The of Haydn’s painting Haydn’s68painting bars wasklingTrumpet3TrumpetwasTrumpetwaskling min divided divideddividedbright andbright Concerto ConcertoConcerto 29 yellowinto intoyellowinto s. Thefour fourfour and andofand of userlines, thelines, lines, theplayed playedplayed stars, experience stars,and andand as asworkedasa workedworked alight a aalight strong, strong,strong, evaluation particle-like withparticle-like withwith cl clcl ear16 ear 16ear16 bars bars melodybars melodymelody rate melody permelody perper from line, line,in line,inin wasthe nine thethe wasproducing producingproducing mid-range. mid-range.extractedmid-range. blindextracted people a aa total from total total fromThe TheThe of was of Haydn’spaintingof paintingpainting Haydn’s68 6868 84%,bars barsbars playedTrumpetwasTrumpetwas divided divided in Concerto3 Concerto min into into and four fourand 29and lines, s. lines,played playedThe and anduser as workedas worked aexperience a strong, strong, with with cl cl earevaluation16evaluation ear16 bars melody barsmelody per per rate rateline,in line,in the fromfromthe producing producing mid-range. mid-range. ninenine blindblind a atotal totalThe Thepeoplepeople of paintingof painting 68 68 waswasbars bars playedTrumpetwasandwasplayedwasplayedTrumpet divided thedivideddivided in inin user Concerto3 3Concerto3 min minmininto intointo experience and andandfour fourfour and 29and 29 29lines, lines, lines, s. playeds. s. playedThe scores TheThe and andand user user useras worked fromasworkedworked aexperience experience aexperiencestrong, strong, eight with withwith cl sighted cl ear16evaluation 1616 evaluationearevaluation bars barsbarsmelody melody participantsper perper line, rate line, inline,rate ratein the fromthe producing fromproducingproducingfrom mid-range. mid-range. were nine ninenine79% blind blinda blind aa total totaltotal The and Thepeople peoplepeople of of ofpainting 80% painting68 6868 bars was bars barswas forwas 84%,wasplayedwasplayed divided and divided in in the 3 3min intousermin into and four andexperience four 29 29lines, lines,s. s. The The and andscores user user worked worked experience fromexperience with eight with 16 evaluationevaluationsighted16 evaluation bars bars per perparticipants line,raterate line,rate fromproducingfrom producingfrom wereninenine nine blind blind79% ablind a total total and peoplepeople peopleof of 80% 68 68 bars was was bars forwas 84%,wasplayedtheplayed84%,played84%,was Classical divided and divided andand in inin the 3 the3the3 min min min intouser and userintouser and and andfour Vivaldiexperience four experienceexperience 29 2929 lines, lines, s. s.s. The schemes,TheThe and andscores userscores userscoresuser worked worked experience experience respectively.experiencefrom fromfrom with eight with eighteight 16 evaluation evaluationsighted16evaluation sightedsighted bars After bars per per oneparticipants participantsparticipants line, rate rate rateline, hour from producingfromfrom producing of practice, ninewere nine ninewerewere blind blind 79%blind a 79% 79% atotal total the and peopleand peopleandpeople of cognitive of 80% 68 80%80% 68 bars was was wasbarsfor forfor theplayed84%,played84%, Classical and and in in the3 the3 min min anduser user and andVivaldi experience experience 29 29 s. s. Theschemes, The scoresuser scoresuser experience respectively experiencefrom from eight eight evaluation sightedevaluation.. sightedAfterAfter one oneparticipants participants rate ratehourhour from from ofof practice, practice,ninewere nine were blind 79%blind 79% thethe andpeople andpeople cognitivecognitive 80% 80% was wasfor for theplayed84%,success84%,the84%,theplayed Classical Classical Classicaland andand in ratein the 3thethe 3min forminuserand user anduserand threeand and Vivaldi experience VivaldiexperienceexperienceVivaldi 29 29 blind s. s. Theschemes, schemes,Theschemes, people scores scoresuserscores user experience wasrespectively experiencefrom respectively fromrespectivelyfrom 100% eight eighteight forevaluationevaluation sighted evaluationsighted.sighted ..After bothAfterAfter theone participants participantsoneparticipantsone raterate Classicalratehour hourhour fromfrom from of ofof practice,nine werenine werepractice,andwerepractice,nine blindblind Vivaldi 79% 79%blind79% the thethe and peopleandpeopleand people cognitive schemes.cognitivecognitive 80% 80%80% waswas forwas forfor success84%,thethe84%,the ClassicalClassical andClassical and rate the the for userandand userand three experienceVivaldiVivaldi experienceVivaldi blind schemes,schemes, schemes,people scores scores was respectivelyrespectivelyfrom respectivelyfrom 100% eight eight for sighted .sighted. .AfterbothAfter After the participantsoneone participantsone Classical hourhour hour ofof of werepractice,practice,and werepractice, Vivaldi 79% 79% thethe andthe and schemes.cognitivecognitive cognitive80% 80% for for success84%,theHere,thesuccessthesuccess84%, Classical ClassicalClassical and weand rate rate ratethe show the for userforand forand anduser anotherthree three three Vivaldi experienceVivaldiVivaldi experience blind blindblind example schemes, schemes,schemes, people people peoplescores scores of was respectivelycomposing fromwasrespectivelyrespectivelywas from 100% 100% 100%eight eight for for sightedfor a .sighted. . singleAfter bothAfter After bothboth the oneparticipants one musicthe onetheparticipants Classical hourClassical Classicalhourhour piece of ofof practice, were representingpractice,and practice, wereandand Vivaldi 79%VivaldiVivaldi 79% the the theand and cognitive schemes.cognitive colorscognitiveschemes. schemes.80% 80% for infor Here,thesuccessthesuccess Classical Classical we rate showrate forand forand another three threeVivaldi Vivaldi blind blindexample schemes, schemes, people people of composingrespectivelywas respectivelywas 100% 100% for afor. single.After bothAfter both onethemusic onethe Classicalhour Classicalhour piece of of practice,representing andpractice, and Vivaldi Vivaldi the the cognitive schemes.colorscognitive schemes. in Here,thesuccessVansuccessHere,successHere,the Classical Gogh’sClassical we wewe rate raterate show showshow “Thefor forandfor and another three anotherthreeanotherthree starryVivaldi Vivaldi blind blindblind example night”, exampleexample schemes, schemes, people peoplepeople as of ofof shown wascomposing respectivelywas wascomposingcomposingrespectively 100% 100%100% in Table for for fora ..aa single.AfterAfter both3single singlebothAfterboth. the oneonethe musicthe onemusicmusic Classical ClassicalhourhourClassical hour piece piecepiece ofof of practice,practice,representing and representingpractice,andandrepresenting Vivaldi VivaldiVivaldi thethe the cognitivecognitive schemes. colorsschemes. schemes.cognitive colorscolors in inin VansuccessHere,successHere, Gogh’s we werate rate show show “Thefor for anotherthree anotherthree starry blind blind examplenight”, example people people as of ofshown was composingwas composing 100% 100% in Table for fora a singleboth singleboth3. the musicthe music Classical Classical piece piece andrepresenting andrepresenting Vivaldi Vivaldi schemes. colorsschemes. colors in in VansuccessHere,Here,VanHere,Vansuccess Gogh’s The Gogh’s Gogh’s we wewe rate ratepaintingshow showshow “Thefor “The“Thefor another threeanotheranother three starry was starrystarry blind dividedblind example examplenight”, example night”,night”, people people into as as ofas ofof shown wasshownthree composing showncomposingcomposingwas 100% rows100% in inin Table Table Tablefor andfor a aa single singlebothsingle both3. four 3.3. the music columns,themusicmusic Classical Classical piece piecepiece and representing and representing representingand played Vivaldi Vivaldi in 2 schemes.colors min colorsschemes.colors and in inin Here,VanHere,Van TheGogh’s weGogh’s we painting show show “The “The another another wasstarry starry divided example night”,example night”, into as of asof shownthreecomposing showncomposing rows in in Table Tableand a a single single four3. 3. musiccolumns, music piece piece and representing representing played in 2 colorsmin colors and in in 10Here,VanVanVanHere, s. Gogh’sThe Gogh’s TheGogh’s Thewe we painting show colorpainting paintingshow “The “The“The another sounds another starrywas starry starrywaswas divided divided individed example night”, night”,examplenight”, each into entryinto asinto asasof of shown shownthreecomposingshown threethreecomposing in the rows rows inrows inin color Table TableTable and a andand asingle mapsingle 3.four 3.3. fourfour table musiccolumns, columns,musiccolumns, were piece piece and and joinedand representing representing played playedplayed to form in inin 2 2 2 colorsmin a mincolorsmin single and andand in in 10VanVan s. Gogh’sThe Gogh’sThe paintingcolor painting “The “The sounds starrywas starry was dividedin dividednight”, night”,each intoentry asinto as shown threeshown threein the rows in rowsin co Table Tablelorlor and and mapmap 3. four3. four tabletable columns, columns, werewere and joined joinedand played played toto formform in in 2 2minaa min singlesingle and and 10Vanmusic1010Van s. s. s. Gogh’sTheThe TheGogh’sTheThe piece. painting paintingcolor painting colorcolor “The Therefore,“The sounds soundssounds starrywas waswasstarry divided divided whiledividedin innight”,in night”,each eacheach listening into entryinto into asentryentry as shown three shownthreethree in in toin the music,therowsthe rowsrowsin in co Tableco coTable lorand lorandandlor the map map3.four map fouruserfour3. table columns, table tablecolumns,cancolumns, recallwere werewere and and theandjoined joinedjoined colorplayed playedplayed to toto corresponding form formin form inin 2 22 min a minmin aa single singlesingle and andand music1010 s. s. TheThe Thepiece.The painting colorpainting color Therefore, sounds sounds was was divided dividedwhilein in each each listening into entryinto entry three three in into the rows themusirows co co andlorlorc, andlor the mapmap fourmap four user tabletablecolumns, tablecolumns, can were were recallwere and andjoinedjoined thejoined played played color toto to formform incorrespond-form in 2 2 min aamin a singlesingle single and and music10to10music10music each s. s.s. The The The Thepiece. piece. soundpiece. painting color colorpaintingcolor Therefore, Therefore,Therefore, and sounds soundssounds was the was location divided in dividedwhilein in whilewhile each eacheach listening of listeningintolistening entry entryintoentry the three colorthree in inin to tothe to thetherows inmusi rows musimusi co thecoco lorandlorc,lor c,and painting.c, the mapthe themap mapfour four user useruser table tablecolumns,table columns, Thesecan cancan were were wererecall recallrecall sound and joined andjoinedjoined the thethe played cuesplayed color color color to toto were form form formincorrespond- correspond-incorrespond- 2 prepared 2 min a amina single singlesingle and and inging10music10music s. totos. The The eacheachpiece. piece. color color soundsound Therefore, Therefore, sounds sounds andand in thethe whilein while each eachlocationlocation listening entrylistening entry ofof in in thethe tothe tothe musi colorcolor musico colorc,lor c, inin themap the map thethe user user table painting.painting.table can can were wererecall recall TheseThese joined joined the the colorsoundsound colorto to form formcorrespond- correspond-cuescues a a single wereweresingle ing10musicformusicingmusicing10 s. the tos. totoThe Thepiece. each piece.comparisonpiece.eacheach color color sound sound Therefore, soundTherefore,Therefore, sounds sounds and purposeandand in thewhile whiletheinwhilethe each each location (aslocationlocation listening listening listeningentry weentry shall of ofinof in theto tothetothe explain the musi musimusi color colorcocolor colorlorc,lorc,c, in inthe theinmapmapthein map the the the userthe useruser system tabletable painting. tablepainting.painting. can cancan werewere recallwere usabilityrecallrecall These ThesejoinedjoinedThese joined the thethe color score color soundcolor soundsoundtoto to formform form correspond-test correspond-correspond- cues cuescues section).aa a singlesingle weresingle werewere preparedmusicingingmusicing toto to piece.each each piece.each for soundsound Therefore, thesoundTherefore, comparison andand and whilethethe whilethe locationlocation location purposelistening listening ofof of (astothethe tothe musi we musicolorcolor color shallc,c, theinin thein explainthethe userthe user painting.painting. painting.can can in recall therecall TheseThese systemThese the the color soundsoundcolor sound usability correspond- correspond- cuescues cues score werewere were musicingpreparedingingpreparedmusic to toto piece.each eacheachpiece. for for sound soundsoundTherefore, the Therefore,the comparison comparison and andand the whilethe thewhile location locationlocation listeningpurpose listeningpurpose of ofof the(as to the the(asto musi we musicolor colorwecolor shallc, shallc, in thein inthe the explain thetheuserexplain user painting. painting.painting. can can in inrecall recallthe the These These Thesesystem thesystem the color sound soundcolorsound usability usability correspond- correspond- cues cuescues scorewere werewerescore ingpreparedpreparedpreparedingprepared to to each each forfor for forsound sound thethe thethe comparisoncomparisoncomparison comparisonand and the the location location purposepurpose purposepurpose of of (asthe(as (asthe(as wewecolor wecolorwe shallshall shallshall in in the explainexplain theexplainexplain painting. painting. inin inin thethe thethe These These systemsystem systemsystem sound sound usabilityusability usabilityusability cues cues scorewerescore scorewerescore testpreparedtestpreparedpreparedtest section).section). section). for forfor thethethe comparison comparisoncomparison purpose purposepurpose (as (as(as we wewe shall shallshall explain explainexplain in inin the thethe system systemsystem usability usabilityusability score scorescore preparedtestingingtestprepareding tosection).to section).to eacheach each for for soundsound soundthe the comparison comparisonandand and thethe the locationlocation location purpose purpose ofof of (asthethe (asthe we colorcolor wecolor shall shall inin in thetheexplain theexplain painting.painting. painting. in in the the TheseThese Thesesystem system soundsound sound usability usability cuescues cues scorewere were scorewere testtesttest section).section). section). preparedtesttesttestprepared section). section).section). for for the the comparison comparison purpose purpose (as (as we we shall shall explain explain in in the the system system usability usability score score testtest section). section). testtest section). section).

Electronics 2021, 10, 1064 7 of 24 Electronics 2021, 10, x FOR PEER REVIEW

Table 3. Color map table and single music coding colors in Van Gogh’s “The starry night” (the sound Table 3. Color map table and single music coding colors in Van Gogh’s “The starry night” (the ﬁlessound developed files developed in this researchin this research are provided are provided separately separately as a Supplemental as a supplement Material). material).

starry night-jdcho@skku.

starstar (Y-L) (Y-L) star (Y-L)star (Y-L) starstar (Y-L) (Y-L) aa littlelittle dark dark windwind (B-L) (B-L) moonmoon (O-S) (O-S) windwind (B-L) (B-L) windwind (B-L) (B-L) (Snare(Snare drum) drum) treetree (G-D) (G-D) star (Y-L) star (White) wind (B-L) stars (Y-L) a little dark treestar (G-D) (Y-L) windstar (B-L) (White) starswind (Y-L) (B-L) treesstars (G-D) (Y-L) (Snarea little drum) dark dark town (G-S) tree (G-D) town (G-S) town (G-S) tree (G-D) wind (B-L) stars (Y-L) trees (G-D) (Snare(Timpani) drum) dark town (G-S) tree (G-D) town (G-S) town (G-S) Bartolome et al. [27] expressed color and depth (advancing and retreating)(Timpani) as temperature and demonstrated the temperature-sensitive replica of Marc Rosco’s work using a thermoelectricBartolome Peltier et al. [27] element expressed and a controlcolor and board. depth For (advancing the sighted and people, retreating) at some as point tem- ofperature the test, and eight demonstrated of the ten users the temperature-sensitive agreed that the conceptual replica dichotomies of Marc Rosco’s warm-near work using and cold-fara thermoelectric were correlated. Peltier element This fact and was a control proven board. during For the the test sighted with people, persons at with some visual point impairment,of the test, eight where of the 83% ten of users the users agreed (a totalthat the of five conceptual out of six) dichotomies linked in bothwarm-near stages and the warmcold-far temperature were correlated. to the concept This fact of was being proven near something, during the and test the with cold persons temperature with visual to the conceptimpairment, of being where far [83%27]. of the users (a total of five out of six) linked in both stages the warmUsing temperature an implicit to the associations concept of test, being Anikin near etsomething, al. [28] confirmed and the thecold following temperature cross- to modalthe concept correspondences of being far [27]. between visual and acoustic features. Pitch was associated with colorUsing lightness, an implicit whereas associations loudness mapped test, Anikin onto greateret al. [28] visual confirmed saliency. the The following hue of colorscross- withmodal the correspondences same luminance between and saturation visual wasand notacou associatedstic features. with Pitch any ofwas the associated tested acoustic with features,color lightness, except whereas for a weak loudness preference mapped to match onto higher greater pitch visual with saliency. blue (vs. The yellow). hue of colors withIn the the same Doppler luminance effect, and the saturation response was of sound not associated waves to with moving any of bodies the tested is illustrated acoustic infeatures, the example except offor the a weak sounding preference of the to locomotive match higher whistle pitch of with a moving blue (vs. train. yellow). When the trainIn blows the Doppler its whistle effect, while the it response is at rest of in sound the station, waves stationary to moving listeners bodies whois illustrated are either in aheadthe example of the engineof the sounding or behind of it the will locomotive hear the same whistle pitch ofmade a moving by the train. whistle, When but the as train the train advances, those who are ahead will hear the sound of the whistle at a higher pitch. blows its whistle while it is at rest in the station, stationary listeners who are either ahead Listeners behind the train, as it pulls further away from them, hear the pitch of the whistle of the engine or behind it will hear the same pitch made by the whistle, but as the train begin to fall [29]. Using such a principle, depth information of a color object located close to advances, those who are ahead will hear the sound of the whistle at a higher pitch. Lis- the viewer’s gaze can be expressed by voice. In other words, if you increase the intensity of teners behind the train, as it pulls further away from them, hear the pitch of the whistle the voice, the color will feel close and intense. However, if you weaken the sound intensity, begin to fall [29]. Using such a principle, depth information of a color object located close the color will feel pale and distant. to the viewer’s gaze can be expressed by voice. In other words, if you increase the intensity Jonas et al. [30] and Cogan et al. [31] found that a higher value in lightness is associated of the voice, the color will feel close and intense. However, if you weaken the sound in- with higher pitch. Another way to match color and sound is to associate an instrument’s tensity, the color will feel pale and distant. tone with color, as in Kandinsky [32]. A low-pitched cello has a low-brightness dark blue Jonas et al. [30] and Cogan et al. [31] found that a higher value in lightness is associ- color, a violin or trumpet-like instrument with a sharp tone feels red or yellow, and a high-pitchedated with higher flute pitch. feels Another like a bright way andto match saturated color sky and blue. sound As is shown to associate in Table an1 instru-, it is possiblement’s tone to compare with color, the sensibilityas in Kandinsky felt in each[32]. instrumentA low-pitched tone cello with has the sensibilitya low-brightness felt in color.dark blue Chroma color, has a violin a relationship or trumpet-like with sound inst intensityrument with [33,34 a ].sharp This tone color feels encoding red or problem yellow, isand the a samehigh-pitched as finding flute the feels entropy like introduceda bright and in saturated Claude Shannon’s sky blue. As Theory shown of Information.in Table 1, it is possible to compare the sensibility felt in each instrument tone with the sensibility felt

Electronics 2021, 10, 1064 8 of 24

Entropy is the average (estimated) minimum resources required to provide information in an event. Marks et al. [35] found that children of all ages and adults matched pitch to value and loudness to chroma. The value (i.e., lightness) is high and heavy, dependent on the light and dark levels of the color. Using the same concept in music, sound is divided into light and heavy feelings according to the high and low octaves of a scale. When the intensity of the sound is strong, the color sensed is close and sharp, whereas when the intensity of the sound is weak, the color becomes distant and muted [35]. Wilms and Oberfeld [36] observed that the increase of the arousal ratings from low to medium to high brightness is only present at the two higher saturation levels (among three levels). For the arousal dimension, it ranges from a relaxed, sleepy figure to an excited, wide-eyed figure. By Rowe [37], warm sound has a tilt towards the bass frequencies. The bass and vocals are more prominent, “bright” is the opposite of warm. Bright gear is better at reproducing high-pitched sounds. Synesthete prevalence among fine-art students was estimated to be 23% [38]. Among various synesthesia due to color (N = 365), colored graphemes are the most common form occurring in two-thirds (66.8%) of a group of 365 synesthetes, and colored time units 19.2%, colored musical sounds 14.5%, and colored general sounds 12.1% [39]. Chromesthesia or sound-to-color synesthesia is a type of synesthesia in which sound involuntarily evokes an experience of color, shape, and movement [40]. Synesthetes that perceive color while listening to music experience the colors in addition to the normal auditory sensations. In Reference [41], synesthetes and non-synesthetes alike associate high-pitched sounds with lighter or brighter colors and low-pitched sounds with darker colors. For some individuals, chromesthesia is only triggered by speech sounds, while others’ chromesthesia can be triggered by any auditory stimuli [42]. In a study investigating variability within categories of synesthesia, 40% of subjects with chromesthesia for spoken words reported that voice pitch, accent, and prosody influenced the synesthetic color [43].

2.3. Review on Color Association with Other Senses Areas that are relatively high in chroma and lightness can seem to “come forward” in the sense of being visually more insistent than other areas, and orange-red, orange, and yellow paints attain higher chroma-lightness combinations than paints of other hues [44]. We can see through experience, that lighter, cooler colors seem to recede, thus making a room feel larger (giving it more “room”), while warmer, more saturated, and darker colors seem to advance, and take up more space in a room, thus making it appear smaller [45]. Ludwig and Simner [46] found that smooth, soft, and round stimuli tended to induce brighter colors, compared to rough, hard, and spiky stimuli. In Slobodenyuk et al. [47], participants were asked to use colors to describe vibro- tactile stimuli of varying frequencies and intensities, simulating variations in roughness– smoothness, heaviness–lightness, elasticity–inelasticity, and adhesiveness–non-adhesiveness. Analysis of the hue, chroma, and brightness of the chosen colors showed a bias towards the red, violet, and blue spectra of hue for the highest intensity haptic stimuli, and toward yellow and green for the lowest intensity, for which green colors were chosen the least. The least intense stimuli also had the lowest level of chroma and highest level of brightness, whereas the opposite was true for the most intense stimuli [47]. There are also indications of hedonic scores mediating cross-modal interactions of odors and colors. Namely, bright colors tend to be rated as pleasant, while darker colors tend to be found more unpleasant (Maric and Jacquot [48]). Strong fragrances are associated with dark colors (Kemp and Gilbert [49]), and there are research results that state that floral fragrances are associated with bright colors (Fiore [50]). Kim [47] found that the floral and woody families showed more distinguishable opposite patterns in both hue and tone parameters: the floral family with brighter warm colors and the woody family with darker (or stronger) cool colors. The warm colors strongly evoked the floral family, while the cool colors the fresh family. The brighter (darker) their Electronics 2021, 10, 1064 9 of 24

lightness values become, the more the ﬂoral (woody) scents are associated. Smoothness, softness, and roundness of stimuli positively correlated with luminance of the chosen color, and smoothness and softness also positively correlated with chroma [51]. These survey results are summarized in Tables4 and5.

Table 4. Semantic differential association between color warmness/coolness and various sensations.

The Semantic Differential Pair Relevant to Source Warmness and Coolness (Sensation) Jonas et al. [30] Strong~Week (sound) Marks et al. [35] Near~Far (thermal-tactile) Bartolomé et al. [18] Warm~Cool (vision) N/A

Table 5. Semantic differential association between color lightness and various sensations.

The Semantic Differential Pair Relevant to Sources Lightness and Darkness (Sensation) Ludwig and Simner [45] Soft~Hard (haptic) Slobodenyuk et al. [46] Round (Bouba)~Pointed (Kiki) Ludwig and Simner [45] (haptic) Ludwig and Simner [45] Smooth~Rough (haptic) Slobodenyuk et al. [46] Light~Heavy (haptic) Slobodenyuk et al. [46] Light~Heavy (sound) Marks et al. [35] Ankins et al. [28] Jonas et al. [30] High~low (sound) Cogan et al. [31] Kandinsky [32] Aroused~Calm (vision) Wilms and Oberfeld [36] Pleasant~unpleasant (scent) Maric and Jacquot [46] Weak~Strong (scent) Kemp and Gilbert [49] Floral~Woody (scent) Fiore [50], Kim [51]

3. Proposed ColorPoetry System This section describes ‘ColorPoetry’, a systematic approach to color expression aids using poetry. The proposed system can explicitly model the color directivity of poems and effectively matches the color appearance dimension (vivid/light/dark/cool) of artwork and poem. Using a user study, we demonstrate that our color-coding scheme can effectively match colors in artwork with poems and provide a signiﬁcantly higher user experience. We envision that this color-coding system utilizing poems will enable many useful applications in appreciating visual arts for the sighted people as well as people with visual impairment.

3.1. Color Selection The perception of color is often described by referring to three dimensions of the color experiences: hue, intensity, and value. There are four visual perceptive terms that are used to describe color appearance, as deﬁned below [44]. Hue: Hue could also be called “root” or “source” color. The hue is always one of the 12 key color places on the basic color wheel (Figure1). Electronics 2021, 10, x FOR PEER REVIEW

applications in appreciating visual arts for the sighted people as well as people with visual impairment.

3.1. Color Selection The perception of color is often described by referring to three dimensions of the color experiences: hue, intensity, and value. There are four visual perceptive terms that are used to describe color appearance, as defined below [44]. Electronics 2021, 10, 1064 Hue: Hue could also be called “root” or “source” color. The hue is always one of10 ofthe 24 12 key color places on the basic color wheel (Figure 1).

Figure 1. 12 RYB color wheel. These images are used under GNU Free Documentation License. Figure 1. 12 RYB color wheel. These images are used under GNU Free Documentation License. Value: The value of a color is always going to be compared against a basic palette of white, grey, or black. A color’s value refers to the quality that distinguishes a light color Value: The value of a color is always going to be compared against a basic palette of from a dark one. In other words, it refers to the lightness or darkness of a hue. white, grey, or black. A color’s value refers to the quality that distinguishes a light color Intensity: Intensity is more commonly called “saturation”. To measure a brightness of from a dark one. In other words, it refers to the lightness or darkness of a hue. any given color, it is ﬁrst necessary to identify the root hue/color it is most closely related Intensity: Intensity is more commonly called “saturation”. To measure a brightness to. This is usually one of the two neighboring colors that sit next to it on the color wheel. of any given color, it is first necessary to identify the root hue/color it is most closely re- Then, the new color’s intensity can be characterized in terms of “brightness” or “dullness” latedas related to. This to theis usually predominant one of rootthe two hue. neighboring colors that sit next to it on the color wheel.Temperature: Then, the new Temperature color’s intensity is perhaps can be the characterized most subjective in terms of the of “brightness” four basic color or “dullness”qualities outlined as related by to Itten. the predominant It is also in this root quality hue. where it is easiest to see the emotional relationshipTemperature: Itten Temperature himself had withis perhaps color. the Temperature most subjective is expressed of the four in basic terms color of warmqual- itiesor cool. outlined by Itten. It is also in this quality where it is easiest to see the emotional rela- tionshipCho Itten et al. himself [26] investigated had with color. that melodies Temperat withure differentis expressed pitch, in tone, terms velocity, of warm and or tempo cool. canCho be used et al. as [26] color investigated sound codes that to melodies easily express with thedifferent color pitch, lightness tone, level. velocity, However, and tempothe number can be of used colors as color that can sound be representedcodes to easily by soundexpress was the limitedcolor lightness to 18 (three level. levels How- of ever,brightness the number for each of colors of six that hues). can be In represented this paper, using by sound poems, was welimited extend to 18 the (three number levels of ofcolors brightness to be expressedfor each of up six to hues). 30, including In this paper, warmer using and poems, cooler colorswe extend of each the hue.number In this of colorsway, theto be visually expressed impaired up to can 30, improveincluding their warmer color and literacy cooler to enjoycolors theof each colorful hue. world In this in way,the painting. the visually In thisimpaired perspective, can improve the extension their color can literacy be done to by enjoy using the sounds colorful to representworld in thesix painting. color hues In (red, this orange,perspective, yellow, the green, extensio blue,n can violet) be done and poemsby using to sounds represent to light/darkrepresent sixand color warm/cool hues (red, colors, orange, simultaneously. yellow, green, In blue other, violet) words, and the poems six color to huesrepresent like red, light/dark orange, andyellow, warm/cool green, colors, blue, and simultaneously. violet on the In 12 other RYB words, color wheel, the six Figure color 1hues, are like represented red, orange, by yellow,sound codes.green, Theblue, six and tertiary violet colors on the are 12 warm RYB and color cool wheel, colors Figure of red, 1, yellow, are represented and blue, such by soundas red-orange codes. The (the six warm tertiary color colors of R), are red-violet warm and (the cool cool colors color of of red, R),yellow-orange yellow, and blue, (the suchwarm as color red-orange of Y), yellow-green (the warm color (the coolof R), color red-violet of Y), blue-green(the cool color (the warmof R), yellow-orange color of B), and blue-violet (the cool color of B), and are represented by sound code and poem (to represent the warmer or cooler colors of the color hue) together. Also, 12 lighter and darker colors from 6 color hues are also represented by sound (color hue) and poem (to represent the

lighter or darker colors of the color hue) together. Also, there are ﬁve achromatic colors: white (WH), black (BK), and three levels of grays (dark gray, middle gray, light gray). In this system, the higher the lightness value, the closer the color is to white, and the lower the lightness, the closer it is to black. As shown in Figure2, the light color (L) has a value of 7 and chroma of 8. The dark color (D) has a value of 3 and a chroma of 8. A saturated color (S) has high chroma (level 15) and colors with the lowest chroma (level 0) are achromatic. Electronics 2021, 10, x FOR PEER REVIEW

(the warm color of Y), yellow-green (the cool color of Y), blue-green (the warm color of B), and blue-violet (the cool color of B), and are represented by sound code and poem (to represent the warmer or cooler colors of the color hue) together. Also, 12 lighter and darker colors from 6 color hues are also represented by sound (color hue) and poem (to represent the lighter or darker colors of the color hue) together. Also, there are five achromatic colors: white (WH), black (BK), and three levels of grays (dark gray, middle gray, light gray). In this system, the higher the lightness value, the closer the color is to white, and the lower the lightness, the closer it is to black. As shown in Figure 2, the light color Electronics 2021, 10, 1064 (L) has a value of 7 and chroma of 8. The dark color (D) has a value of 3 and a chroma11 of 24of 8. A saturated color (S) has high chroma (level 15) and colors with the lowest chroma (level 0) are achromatic.

FigureFigure 2.2. SaturatedSaturated (S),(S), lightlight (L),(L), andand darkdark (D)(D) forfor red.red.

3.2.3.2. AlgorithmAlgorithm ofof PoetryPoetry CodingCoding ColorsColors In this section, we explore the poem coding colors representing five color appearance In this section, we explore the poem coding colors representing five color appearance dimensions. First, we need to identify the poems that feels “good” for the color appearance dimensions. First, we need to identify the poems that feels “good” for the color appear- dimension used in the visual artwork. Next, we express such color appearance dimension ance dimension used in the visual artwork. Next, we express such color appearance di- using three levels of voice pitches respectively, while reciting a poem. Therefore, combining mension using three levels of voice pitches respectively, while reciting a poem. Therefore, voice sounds and poems in this way makes it simpler and easier to identify color, including combining voice sounds and poems in this way makes it simpler and easier to identify light, dark, warm, and cool colors, compared to the conventional way of using a single color, including light, dark, warm, and cool colors, compared to the conventional way of modality such as sound, tactile pattern, or temperature alone. Given an artwork image A using a single modality such as sound, tactile pattern, or temperature alone. Given an and a set of poems DP, as illustrated in Table6, the goal is to find the set of poems P ∈ DP artwork image A and a set of poems DP, as illustrated in Table 6, the goal is to find the set that are mostly relevant to the color appearance dimension directivity of the image. Let of poems P ∈ DP that are mostly relevant to the color appearance dimension directivity S(A) (respectively S(P)) be denoted as a measure of the overall color appearance dimensions of the image. Let S(A) (respectively S(P)) be denoted as a measure of the overall color observed in artwork A (respectively P). We are now ready to describe the ColorPoetry Algorithmappearance 1 asdimensions follows. The observed purpose in ofartwork the algorithm A (respectively is to find P). the We set are of “good”now ready poems to de- P withscribe the the maximum ColorPoetry significance Algorithm of Art–Poem1 as follows. similarity The purpose score of denoted the algorithm by sigmax is to S(A, find P): the set of “good” poems P with the maximum significance of Art–Poem similarity score denoted by sigmax S(A, P): Algorithm 1 ColorPoetry (i.e., Poetry Coding Colors). InputAlgorithm: a color 1 CColorPoetry in an artwork (i.e., A, a Poetry poem database Coding DP.Colors). ∈ OutputInput:: a a color set of C poems in an P artworkDP. A, a poem database DP. 1: For every color in C Output: a set of poems P ∈ DP. Identify the set of poems P, finding sigmax S (A, P), for all P ∈ DP. 2:1: ReturnFor every P. color in C Identify the set of poems P, finding sigmax S (A, P), for all P ∈ DP. 3.3.2: Return Overall P. Solution Strategy There are three main steps to solving the problem. 3.3. Overall Solution Strategy (1) Determine poem database. Color poems are about a single color, using descriptions, nouns,There and are other three key main elements steps to expresssolving the feelings problem. about that color. An easy format for this type(1) Determine of poem is poem describing database. the color Color using poems the are five about senses: a single looks, sounds,color, using tastes, descrip- feels, andtions, smells. nouns, From and Wikipedia,other key elements lyric poetry to expre is a formss feelings of poetry about that that does color. not An attempt easy format to tell a story, as do epic poetry and dramatic poetry, but is of a more personal nature instead. Rather than portraying characters and actions, the lyric poet addresses the reader directly, portraying his or her own feelings, states of mind, and perceptions. In the poems, colors are

used as a metaphor or to express the context in the poem, and we mainly collected them extensively through browsing relevant websites like “poemhunter.com”[48], and literature. The color database we built consists of 12–15 poems with different moods for each of the 6 hues: red, orange, yellow, green, blue, purple, including white, grey, and black. These poems are selectively analyzed and selected according to the appearance dimensions of the color in a given artwork. (2) Determine color perceptual adjectives. The perceptual semantic difference method is used to establish a rigorous sensory analysis process to evaluate algorithms to ﬁnd phrases in poetry that match the color of a given artwork. To reﬂect the user’s perception Electronics 2021, 10, 1064 12 of 24

of color perception, it is necessary to use a set of color-related perceptual adjectives that have been validated in psychological research. The amount and ﬁdelity of the collection of perceptual adjectives affects the quality of the research data. Adjectives that describe colors have been extensively collected through the existing literature. (3) Data analysis and user experience test. The results of the evaluation of the correla- tion between the perceptual meanings of the colors that appeared in artwork and poetry were analyzed with statistical tools. The user experience test of the proposed color-coding system will be performed through user interviews.

3.4. Voice Modulation Four color properties such as light/dark/warm/cool can be expressed by voice, so the brightness of the color is modulated according to the pitch of the sound, and the degree of warmth and coolness of the color can be expressed by the strength and reverberation of the voice. A voice was produced using adobe audition. To express the lighter color in voice, the pitch was raised by 2.5 semitones from the original voice. To express darker color in voice, the pitch was lowered by 5 semitones from the original voice. To express vivid (i.e., highly intense) color in voice, the speed was stretched to 75% from the original voice and made it faster. To express warmer color in voice, a stereo-reflection plate was set among presets at full reverb and amplified +2.5 dB from the original voice. Finally, to express cool color in voice, “deserted room” was set among presets in full reverb. Two experimental artworks are illustrated to evaluate the usability of the appreciation method for the colors of artworks for the visually impaired that have been confirmed through such a test. Two versions of voice modulation were made and used according to preference: (1) the first version (called “word”) with the corresponding voice modulation applied for each color word, and (2) the second version (called “phrase”) in which the words contained in the same line related to the color are applied with the same voice modulation corresponding to the color.

3.5. Use Cases Here, Vincent van Gogh’s 1889 work ‘The Starry Night’ (Modern museum of art, NY) and Henri Matisse’s 1910 work ‘Dance’ (Hermitage Museum, Saint Petersburg) are illustrated as use cases. ‘Starry Night’ conceptually and dynamically expresses the ﬂuc- tuating air of an invisible night sky. Matisse used only four colors: blue, green, orange, and red. These vivid hues create an intense contrast. Both of these works well match ElectronicsElectronics 2021 2021, 10, ,10 x ,FOR x FOR PEER PEER REVIEW REVIEW the conceptual elements of color, so they are suitable for testing the usefulness of how to ElectronicsElectronics 2021 2021, 10, ,10 x ,FOR x FOR PEER PEER REVIEW REVIEWappreciate works of art using multiple senses. One of two poems per color were selected

from about 2000 color poems in the “poem hunter website” [52], as shown in Tables6 and7 . TableTable 6. Poetry6. Poetry for for representing representingThe table colors colors also in inVan shows Van Gogh’s Gogh’s the vocal“The “The starry narration starry night” night” for (the coding (the sound sound each files files color developed developed in words in inthis andthis research tones.research are are providedprovided separately separately as asa supplement a supplement material). material). TableTable 6. 6.Poetry Poetry for for representing representing colors colors in inVan Van Gogh’s Gogh’s “The “The starry starry night” night” (the (the sound sound files files developed developed in inthis this research research are are Tableprovidedprovided 6. Poetry separately separately for representing as asa supplement a supplement colors material). in material). Van Gogh’s “The starry night” (the sound ﬁles developed in this research are PartPart of ofPoem Poem Excerpt Excerpt ModulatedModulated Voice Voice providedColorColor separately as aPoet SupplementalPoet Material). fromPartfromPart the of the of PoemOriginal PoemOriginal Excerpt Excerpt Poem Poem WordWordModulatedModulated Voice PhraseVoicePhrase ColorColor PoetPoet Part of Poem Excerptfrom the Modulated Voice Color Poet fromfrom the the Original Original Poem Poem Word Phrase DarkDark blue blue sky’sOriginal sky’s out out Poem there, there, Word PhrasePhrase DarkBeforeBefore blue daybreak sky’sdaybreak out is there, isin; in; P1:P1: Dark Dark Blue Blue Sky Sky DarkDark blue blue sky’s sky’s out out there, there, FallingFalling starsBefore stars here here daybreak and and there, isthere, in; P1: Dark Blue Sky BeforeBefore daybreak daybreak is isin; in; P1.mp3P1.mp3 P1.mp3P1.mp3 PeterP1:PeterP1: Dark S. Dark S.Quinn BlueQuinn Blue Sky[53] Sky[53] Falling stars here and there, Peter S. Quinn [53] FallingNightFallingNight starsand starsand distant here distanthere and moon,and moon,there, there, CoolCool dark dark Peter S. Quinn [53] Night and distant moon, P1.mp3P1.mp3 P1.mp3P1.mp3 Cool dark blue sky Peter S. Quinn [53] SleeplessSleepless is ismy my night; night; NightNight andSleepless and distant distant is mymoon, moon, night; CoolblueCoolblueand darksky dark nightsky SleeplessThe is skiesmy night; so blue, andand night night TheSleeplessThe skies skies sois so myblue, blue, night; blueblue sky sky The Sky, the Ocean, sadly Blue. P2: Blue on Blue TheThe Sky, Sky,The the theskies Ocean, Ocean, so blue,sadly sadly Blue. Blue. andand night night P2:P2: Blue Blue on on Blue Blue UponThe askies blue-lit so tinglingblue, Star, Sandra Feldman [54] Upon a blue-lit tingling Star, TheTheUpon Sky,That Sky, thea Love blue-litthe Ocean, canOcean, traveltingling sadly sadly to theBlue. Star, Blue. moon, P2.mp3P2.mp3 P2.mp3P2.mp3 SandraSandraP2:P2: Blue Feld Blue Feld onman on manBlue Blue [54] [54] ThatThatUpon UponLoveAnd Love a canblue-lit mucha blue-litcan travel that’stravel tingling tinglingto Blue tothe the is Star,moon, a Star,moon, Mirage. SandraSandra Feld Feldmanman [54] [54] P2.mp3P2.mp3 P2.mp3P2.mp3 AndThatAndThat much Love muchLove canthat’s canthat’s travel travelBlue Blue to is to theisa the Mirage.a moon, Mirage. moon,

AndAnd much muchA that’s Ayellow that’s yellow Blue Bluestar staris isa Mirage. a Mirage. LightLight warm warm A yellowIn Ina skya sky star P3:P3: Yellow Yellow Star Star A yellow star yellow star Twinkling white ones LightLightyellow warm warm star TwinklingInIn a skya whitesky ones P3.mp3P3.mp3 P3.mp3P3.mp3 P3:VijayP3:Vijay Yellow Yellow Sai Sai [55] Star [55] Star yellowWhiteyellowWhite star star starstar TwinklingTwinklingIt’sIt’s a yellowa yellow white white bird birdones ones VijayVijay Sai Sai [55] [55] P3.mp3P3.mp3 P3.mp3P3.mp3 WhiteWhite star star MyMy beautiful beautifulIt’sIt’s a yellowa little yellow little yellow bird yellow bird bird! bird!

MyMy beautiful beautiful little little yellow yellow bird! bird! P4:P4: The The wind wind has has a a TheThe wind wind has has a bluea blue tail tail P4: Theblueblue wind tail tail has a P4: The wind has a TheTheThe wind wind wind also also has has hasa ablue whitea white tail tail tail P4.mp3P4.mp3 P4.mp3P4.mp3 ArjenArjen blueDuinker Duinker tail [56] [56] The wind has a blue tail LightLight Cool Cool blue tail The wind also has a white tail P4.mp3P4.mp3 P4.mp3P4.mp3 Arjen Duinker [56] The wind also has a white tail blueLightblue wind Cool wind Arjen Duinker [56] Light Cool P5:P5: Blue, Blue, Blue Blue and and BlueBlue smells smells like like my my mother’s mother’s perfume perfume blueblue wind wind Blue P5:P5: Blue, Blue, BlueBlue Blue and and P5.mp3P5.mp3 P5.mp3P5.mp3 BlueBlue smells smellsOn likeOn likea windymya windymy mother’s mother’s day day perfume perfume KarunKarunBlue AkerBlue Aker [57] [57] OnOn a windya windy day day P5.mp3P5.mp3 P5.mp3P5.mp3 KarunKarun Aker Aker [57] [57] CrushedCrushed orange orange moon, moon, P6:P6: Crushed Crushed Orange Orange HoldHoldCrushed your your color colororange for for the moon, the delight delight WarmWarm MoonMoon Crushed orange moon, P6:P6: Crushed Crushed Orange Orange of night. HoldHold your your color colorof night. for for the the delight delight P6.mp3P6.mp3 P6.mp3P6.mp3 orangeorangeWarmWarm moon moon RobertRobert MurrayMoon MurrayMoon Smith Smith WhenWhen you you crushof crush ofnight. night.orange orange light light for for orangeorange moon moon RobertRobert Murray[58] Murray[58] Smith Smith P6.mp3P6.mp3 P6.mp3P6.mp3 WhenWhen you you crushour crushour sight. orangesight. orange light light for for [58][58] ourour sight. sight. P7P7: Beneath: Beneath the the Dark Dark I thinkI think of ofus us beneath beneath the the trees, trees, CoolCool dark dark P7: BeneathGreenGreen Grove theGrove Dark P7: Beneath the Dark BeneathI thinkBeneath of the us the dark beneath dark green green the grove. trees, grove. P7.mp3P7.mp3 P7.mp3P7.mp3 GreenGreen MeganMegan Cooper Cooper [59] [59] I think of us beneath the trees, CoolCool dark dark GreenGreen Grove Grove P7.mp3 P7.mp3 (or Cool BeneathBeneath the the dark dark green green grove. grove. P7.mp3 P7.mp3 Green(orGreen Cool MeganMegan Cooper Cooper [59] [59]

(orbrown)(orbrown) Cool Cool P8P8: Color: Color Poems; Poems; BrownBrown in inthe the color color of ofnature nature itself. itself. Brown Brown brown)treetree Brown is the color of an oakwood shelf. brown) P8P8: Color: ColorBrown Poems; Poems; BrownBrownis in the inthe thecolor color color of of an ofnature oakwoodnature itself. itself. shelf. Brown Brown P8.mp3P8.mp3 P8.mp3P8.mp3 treetree AlexAlex SaffordBrown SaffordBrown [60] [60] isBrown istheBrown the color iscolor isthe ofthe trunks ofan trunksan oakwood oakwood of ofthe the treesshelf. treesshelf. P8.mp3P8.mp3 P8.mp3P8.mp3 AlexAlex Safford Safford [60] [60] BrownBrown is isthe the trunks trunks of ofthe the trees trees

CoolCool green green P9P9: Green:: Green: The The Color Color MyMy favorite favorite colors colors are are green green and and brown. brown. P9.mp3P9.mp3 P9.mp3P9.mp3 CoolCooltown towngreen green P9BriP9: Green:Bri: EdwardsGreen: Edwards The The [61]Color [61]Color MyGreen-leafedMyGreen-leafed favorite favorite colors treescolors trees arepolka-dot arepolka-dot green green and our and our brown.town. brown. town. towntown BriBri Edwards Edwards [61] [61] Green-leafedGreen-leafed trees trees polka-dot polka-dot our our town. town. P9.mp3P9.mp3 P9.mp3P9.mp3

Electronics 2021, 10, Electronicsx FOR PEER 2021 REVIEW, 10, x FOR PEER REVIEW Electronics 2021, 10, Electronics x FOR PEER 2021 REVIEW, 10, x FOR PEER REVIEW

Electronics 2021, 10, Electronicsx FOR PEER 2021 REVIEW, 10, x FOR PEER REVIEW

ElectronicsTable 2021 6., 10Poetry, Electronicsx FOR forTable PEER representing 2021 REVIEW6., 10Poetry, x FOR colorsfor PEER representing in REVIEW Van Gogh’s colors “The in Van starry Gogh’s night” “The (the starry sound night” files developed (the sound in files this developed research are in this research are Electronics Tableprovided 2021 6., 10Poetry separately, Electronics x FOR forTableprovided PEER representing as2021 REVIEWa6. ,supplement 10Poetry separately, x FOR colorsfor PEER representing material).as in REVIEWa Vansupplement Gogh’s colors “Thematerial). in Van starry Gogh’s night” “The (the starry sound night” files developed (the sound in files this developed research are in this research are

ElectronicsprovidedTable 2021 6., Poetry10separately, Electronicsx FOR forprovidedTable PEER representing as 2021 a 6.REVIEW supplement, Poetry10separately, x FOR colorsfor PEER representing material).as in a REVIEW supplementVan Gogh’s colors material).“The in Van starry Gogh’s night” “The (the starry sound night” files developed (the sound in files this developed research are in this research are Tableprovided 6. Poetry separately forTableprovided representing as a6. supplement Poetry separately colorsfor representing material).as in a Vansupplement Gogh’s colorsPart “Thematerial). ofin Van Poemstarry Gogh’s Excerptnight”Part “The of(the Poemstarry sound Excerptnight” files developed (the soundModulated in files this developed research VoiceModulated are in this research Voice are TableColor 6. Poetry forTableColor representing Poet6. Poetry colorsfor representing inPoet Van Gogh’s colors “The in Van starry Gogh’s night” “The (the starry sound night” files developed (the soundModulated in files this developed research VoiceModulated are in this research Voice are provided separatelyprovided as a supplement separately material).as a supplement fromPart material). theof Poem Original ExcerptfromPart Poem theof Poem Original Excerpt Poem TableColor 6. Poetry forTableColor representing Poet6. Poetry colorsfor representing inPoet Van Gogh’s colors “The in Van starry Gogh’s night” “The (the starry sound night” files developed (theWord sound in files this developed researchWordPhrase are in this researchPhrase are provided separatelyprovided as a supplement separately material).as a supplement fromPart material). the of PoemOriginal fromExcerptPart Poem the of PoemOriginal Excerpt Poem Word Modulated VoiceWordPhraseModulated VoicePhrase providedTableColor 6. Poetryseparately forprovidedTableColor representingas aPoet 6.supplement Poetryseparately colorsfor material).representingas inaPoet supplement Van Gogh’s colors material). “The in Van starry Gogh’s night” “The (the starry sound night” files developed (the soundModulated in files this developed research VoiceModulated are in this research Voice are DarkfromPart blue theof Poem Originalsky’s Dark ExcerptoutfromPart Poem there, blue theof Poem Original sky’s Excerptout Poem there, Word WordPhrase Phrase Color ColorPoet Poet Modulated VoiceModulated Voice provided separatelyprovided as a supplement separately material).as a supplement DarkfromPart material). blue theof Poem Originalsky’s Dark ExcerptfromoutPart Poem there, blue theof Poem Original sky’s Excerptout Poem there, Word WordPhrase Phrase Color ColorPoet Poet PartBefore of daybreakPoem ExcerptPartBefore is in; of daybreakPoem Excerpt is in; Modulated VoiceModulated Voice Color P1:Color DarkPoet Blue SkyP1: DarkPoet Blue SkyDarkfromBefore bluethe daybreak Original sky’sDark fromoutBefore isPoem there, in;bluethe daybreak Original sky’s out isPoem there, in; Word WordPhrase Phrase P1: Dark Blue SkyP1: Dark BlueFalling SkyfromPart the starsof OriginalPoem hereFalling fromExcerpt andPart Poem there,the starsof OriginalPoem here Excerpt and Poem there, P1.mp3 Modulated P1.mp3VoiceP1.mp3Modulated VoiceP1.mp3 Color PeterColor S.Poet Quinn [53]Peter S.Poet Quinn Falling Dark[53]Before blue stars daybreak sky’s hereFallingDark outandBefore is there, there,bluein; stars daybreak sky’s here outand is there, there,in; Word WordPhrase Phrase PeterP1: Dark S. Quinn Blue Sky[53]PeterP1: Dark S. Quinn Blue DarkSky[53]Nightfrom blue the and sky’sOriginal distantDark outNightfrom moon, Poemthere, blue the and sky’sOriginal distant out moon, Poemthere,P1.mp3 P1.mp3P1.mp3 P1.mp3 Cool dark Cool dark FallingBefore stars daybreak hereFalling andBefore is in;there, stars daybreak here and is in;there, Word WordPhrase Phrase P1: Dark Blue SkyP1: Dark Blue DarkSkyNightSleepless blue and sky’s distant is myDark outNight Sleeplessnight; moon, there, blue and sky’s distant is my out night; moon, there, P1.mp3 P1.mp3P1.mp3 P1.mp3 Cool dark CoolPeter dark S. Quinn [53]Peter S. QuinnFalling [53]Before stars daybreak hereFalling andBefore is in;there, stars daybreak here and is in;there, blue sky P1:blue Dark sky Blue SkyP1: Dark Blue SkyDarkNightSleepless blue and sky’s distantis myDark Nightout Sleeplessnight; moon, there, blue and sky’s distant is my out night; moon, there, P1.mp3 P1.mp3P1.mp3 P1.mp3 Cool dark PeterCool darkS. Quinn [53]Peter S. QuinnFalling [53]Before stars daybreak hereFalling andBefore is there,in; stars daybreak here and is there,in; andblue night sky P1:andblue Dark night sky Blue SkyP1: Dark Blue SkyNightSleepless The and skies distant is somyNight blue, Sleeplessnight; moon,The and skies distant is somy blue, night; moon, P1.mp3 P1.mp3P1.mp3 P1.mp3 Cool dark PeterCool darkS. Quinn [53]Peter S. QuinnFalling [53]Before stars daybreak hereFalling andBefore is there, in;stars daybreak here and is there, in; andblue night sky PeterandP1:blue Darknight S.sky Quinn Blue [53]SkyPeterP1: Dark S. Quinn Blue [53]SkyNightSleepless The and skies distant is mysoNight blue, Sleeplessnight; moon,The and skies distant is myso blue, night; moon, P1.mp3 P1.mp3P1.mp3 P1.mp3 Cool dark Cool dark The Sky, the Ocean,The Sky,sadly the Blue. Ocean, sadly Blue. andblue night sky andblueP2: night Bluesky on BlueP2: Blue on BlueFallingNight The andstars skies distant here soFallingNight andblue, moon,The there, andstars skies distant here so andblue, moon, there, P1.mp3 P1.mp3P1.mp3 P1.mp3 Cool dark CoolPeter dark S. Quinn [53]Peter S. QuinnTheUpon [53] Sky,Sleepless a theblue-lit Ocean, isThe my Upontingling Sky,Sleeplessnight;sadly a theblue-lit Star, Blue. Ocean, is my tingling night;sadly Star, Blue. andblue night sky SandraandblueP2: night Bluesky Feld onman BlueSandra [54]P2: Blue Feld onmanThe Blue Night [54]Sky,Sleepless The the and skies Ocean, isdistantThe myso Nightblue, Sky,Sleeplessnight;sadly moon,The the and Blue.skies Ocean, isdistant myso blue, night;sadly moon, Blue.P2.mp3 P2.mp3P2.mp3 P2.mp3 Cool dark Cool dark Upon a blue-lit Upontingling a blue-lit Star, tingling Star, Electronicsandblue2021 night sky, 10 , 1064 SandraandblueP2: night Bluesky Feld onman BlueSandra [54]P2: Blue Feld onmanThat Blue [54]LoveSleepless The can skies travel Thatis somy blue, Loveto Sleeplessnight; Thethe can moon,skies travel is somy blue, to night; the moon, P2.mp3 P2.mp3P2.mp313 of 24 P2.mp3 ThatTheUpon LoveSky, a thecanblue-lit Ocean, travelThatThe Upontingling Love Sky,tosadly the a thecanblue-lit Star,moon,Blue. Ocean, travel tingling tosadly the Star,moon,Blue. andblue night sky SandraandblueP2: night Blue sky Feld onman BlueSandra [54]P2: Blue Feld onAndmanThe Blue muchSky,[54] The the that’s skies Ocean,AndThe Blueso blue,much Sky,sadly isThe a the Mirage. that’s Blue.skies Ocean, Blueso blue, sadly is a Mirage. Blue.P2.mp3 P2.mp3P2.mp3 P2.mp3 P2: Blue on BlueP2: Blue onAndThat BlueUpon muchLove a blue-litcanthat’s travelAndThat Blue Upontingling much Loveto is thea blue-litMirage.canStar,that’s moon, travel Blue tingling to is thea Mirage.Star, moon,P2.mp3 P2.mp3P2.mp3 P2.mp3 and night Sandraand night Feld manSandra [54] FeldmanTheUpon [54]Sky, Thea theblue-lit skies Ocean,The Upontinglingso Sky,blue,sadly Thea theblue-lit Star, Blue. skies Ocean, tinglingso blue,sadly Star, Blue. P2: Blue on BlueP2: Blue onThatAnd Blue muchLove Acan that’s yellow travelThatAnd Blue star muchLoveto is the a Acan Mirage. that’smoon, yellow travel Blue star to is the a Mirage. moon,P2.mp3 P2.mp3P2.mp3 P2.mp3 Sandra FeldmanSandra [54] FeldmanTheUpon [54] Sky, a Ablue-litthe yellow Ocean,The Upontingling star Sky,sadly a Ablue-littheStar, Blue.yellow Ocean, tingling star sadly Star, Blue. Light warm SandraLightP2: warmBlue Feld onman BlueSandra [54]P2: Blue Feld onAndThatman Blue much[54]LoveTable can 6.that’sCont.In travel AndThata Bluesky muchLoveto is the a can Mirage. that’smoon,In travel a Bluesky to is the a Mirage. moon,P2.mp3 P2.mp3P2.mp3 P2.mp3 P3: Yellow Star P3: Yellow Star A yellow star A yellow star Light warm Light warm AndThatUpon muchLove a canblue-litthat’sIn travel AndThata Bluesky Upontingling much Love to is the aa Mirage.canblue-lit that’sStar,moon,In travel a Bluesky tingling to is the a Mirage. Star,moon,P2.mp3 P2.mp3P2.mp3 P2.mp3 yellow star SandrayellowP3: Yellow starFeld man StarSandra [54] P3: Yellow Feldman Star [54] Twinkling whiteTwinkling ones white ones P3.mp3 P3.mp3P3.mp3 P3.mp3 Lightyellow warm star LightyellowVijay warm star Sai [55] Vijay SaiAndThat [55] much LoveTwinklingPartA ofthat’scan yellow PoemIn travelAnd aThat Bluewhite sky Excerptfrom starmuch Loveto Twinklingis ones thea A Mirage.that’scan moon, yellowIn thetravel a Bluewhite sky star to is ones thea Mirage. moon, Modulated Voice WhiteColor star WhiteP3:Vijay Yellow star Sai [55]Poet Star P3:Vijay Yellow Sai [55] Star It’sA yellowa yellow star bird It’s A yellowa yellow star bird P3.mp3 P3.mp3P3.mp3 P3.mp3 LightyellowWhite warm star star LightyellowWhite warm star star And muchTwinklingIt’s that’s aInOriginal yellow aAnd whiteskyBlue muchbird PoemTwinkling is onesIt’s a Mirage.that’s a In yellow a whiteskyBlue bird is ones a Mirage. Word Phrase Light warm LightP3:Vijay warm Yellow Sai [55] Star P3:Vijay Yellow Sai [55] MyStar beautiful A yellowIn littlea skyMy star yellow beautiful A bird!yellowIn littlea sky star yellow bird!P3.mp3 P3.mp3P3.mp3 P3.mp3 yellowWhite star yellowWhiteP3: Yellow star Star P3: Yellow MyStar beautiful TwinklingIt’s a Ayellow little yellow whiteMy yellow beautifulbirdTwinkling staronesIt’s bird! a yellow little white yellow bird ones bird! P3.mp3 P3.mp3P3.mp3 P3.mp3 Lightyellow warm star LightyellowVijay warm star Sai [55] Vijay Sai [55] TwinklingA Inyellow a whitesky star Twinkling ones A Inyellow a whitesky star ones WhiteLight warmstar WhiteP3: Yellow star Star P3: Yellow MyStar beautiful It’s a yellow littleInMy a skyyellow birdbeautifulIt’s abird! yellow little yellow bird bird!P3.mp3 P3.mp3P3.mp3 P3.mp3 yellowLight warm star yellowLightP4:Vijay The warm starP3: windSai Yellow [55] hasP4: StaraVijay The windSai [55] has Twinkling a In a white sky Twinkling ones In a white sky ones Whiteyellow star star WhiteP4:P3:Vijay The Yellowstar windSai [55] StarhasP4: P3:aVijay The Yellow windSai [55]My Starhas Thebeautiful a It’s windTwinkling a yellow littlehasMy a whiteyellow ThebirdbluebeautifulIt’s wind onestail abird! yellow littlehas a yellow birdblue tail bird!P3.mp3 P3.mp3P3.mp3 P3.mp3 Whiteyellow star star Whiteyellow bluestar starVijay tail Sai [55] blue tail TheTwinklingIt’s wind a yellow has white a ThebluebirdTwinkling onesIt’s wind tail a yellow has white a bluebird ones tail White star P4:Vijay The wind Sai [55] hasP4: aVijay The wind Sai My[55]The has beautiful wind a It’salso little a Myhas yellowThe yellow beautifula windwhite bird bird! also tail little has yellow a white bird! P4.mp3P3.mp3tail P4.mp3P3.mp3P4.mp3P3.mp3 P4.mp3P3.mp3 Arjenblue Duinker tail [56]Arjen blue Duinker tail [56] P4.mp3 P4.mp3P4.mp3 P4.mp3 White star P4:White The star wind hasP4: a The windMyThe has Thebeautiful My winda It’s beautifulwind alsoa yellow littlehas hasMyThe little a yellow aThebeautiful blue bird windwhite yellowIt’s wind tail bird! alsoa tail bird!yellow littlehas has a yellow a bluebird white tail bird! tail Light Cool LightArjen blue CoolDuinker tail [56]Arjen blue Duinker tail [56] P4.mp3 P4.mp3P4.mp3 P4.mp3 Light Cool LightP4: The Cool wind hasP4: a The windMy ThehasThe beautiful winda wind also haslittle hasMyThe a Theyellowblue abeautiful windwhite wind tail bird!also tail haslittle has a yellowbluea white tail bird! tail blue wind blueArjen windblue Duinker tail [56]Arjen blue Duinker tail [56]The wind has a Theblue wind tail has a blue tailP4.mp3 P4.mp3P4.mp3 P4.mp3 Light Cool P4:LightP5: The Blue, CoolP4: windThe Blue windhas andP4:P5: a has The Blue, a wind BlueThe has and winda The also wind hasThe has a windwhite a blue also tail has a white tail blue wind blueArjen windblue Duinker tail [56]Arjen blue Duinker tail [56]The wind has a Theblue wind tail has a blue tailP4.mp3 P4.mp3P4.mp3 P4.mp3 Light Cool LightP4:P5: TheBlue, blueCool wind Bluetail blue has and tailP4:P5: a TheBlue,blue windBlue BluetailThe smellshas and wind a like alsoBlue my hasThe mother’s smells a windwhite like alsoperfume tail my has mother’s a white perfume tail blue wind Arjenblue wind DuinkerBlue [56]Arjen DuinkerBlueBlue smells [56]The windlikeBlue my has mother’s smells a Theblue windlike tail perfume my has mother’s a blue tail perfumeP4.mp3P5.mp3 P4.mp3P5.mp3P4.mp3P5.mp3 P4.mp3P5.mp3 Light Cool LightP5: Blue, CoolArjen Blue Duinker andP5: [Blue,56] BlueThe andtail wind TheOn also winda windy has alsoThe a hasday windwhiteOn a white also atail windy tailhas a day white tail blueLight wind Cool ArjenblueKarun wind blueDuinkerBlue Aker tail [57] [56]ArjenKarun blueDuinkerBlue Aker tail [57] [56] P5.mp3 P5.mp3P5.mp3 P5.mp3 Light Cool LightP5: Blue, Cool Blue andP5: Blue, BlueBlueThe smellsand wind On like alsoa Bluewindymy has Themother’s smells aday windwhiteOn like perfumealsoa tail windymy has mother’s aday white perfume P4.mp3tail P4.mp3P4.mp3 P4.mp3 blueblue wind wind blueArjenKarun wind DuinkerBlue Aker [57] [56]ArjenKarun DuinkerBlue Aker [57] [56] P5.mp3 P5.mp3P5.mp3 P5.mp3 P5: Blue,P5: Blue, Blue Blue andP5: and Blue, BlueBlue Blue Bluesmells and smells On like a likeBlue mywindy mymother’s smells mother’s dayOn like perfume a perfume mywindy mother’s day perfume blueLight wind Cool blueLightKarun wind CoolBlue Aker [57]Karun Blue Aker [57] P5: Blue,Karun Blue Aker andP5: [ 57Blue,] Blue Blue smells andCrushed On like aOnBlue mywindy orange a windymother’s smells Crushedday moon, dayOn like perfume a mywindy orange mother’s day moon, perfumeP5.mp3 P5.mp3P5.mp3 P5.mp3 blue wind P6:blueKarun Crushed windBlue Aker Orange [57]P6:Karun Crushed Blue AkerBlue Orange smells[57]Crushed likeBlue my orange mother’s smellsCrushed moon, like perfume my orange mother’s moon, perfumeP5.mp3 P5.mp3P5.mp3 P5.mp3 P6:P5: Crushed Blue,Blue Blue Orange P6:andP5: Crushed Blue,Blue Blue OrangeHold and your On acolor windyHold for day the yourOn delight acolor windy for day the delight Warm KarunWarmMoon Aker [57]Karun Moon AkerBlueHold [57] smellsCrushed yourOn like acolor Blue windy myorangeHold formother’s smells day Crushedthe moon,yourOn delightlike perfumeacolor windy myorange formother’s day the moon, delight perfumeP5.mp3 P5.mp3P5.mp3 P5.mp3 Warm P6:KarunWarm CrushedMoon BlueAker Orange [57]P6:Karun Crushed Moon BlueAker Orange [57] Crushedof night. orange moon,of night. P6.mp3 P6.mp3P6.mp3 P6.mp3 orange moon orangeRobertP6: moonMurrayCrushed RobertSmith Orange MurrayHold SmithCrushed yourOn ofcolora windyorange night.Hold forCrushed daythemoon, yourOn delight ofcolora windyorange night. for daythemoon, delight P5.mp3 P5.mp3P5.mp3 P5.mp3 Warm P6:KarunWarm CrushedMoon Aker Orange [57]P6:Karun Crushed Moon Aker WhenOrange [57]Hold you yourcrush colorWhen orange for you the light delightcrush for orange lightP6.mp3 for P6.mp3P6.mp3 P6.mp3 orangeWarm moon orangeRobert moonMurray[58] Moon RobertSmith Murray[58] Hold SmithCrushed your color orangeHold forCrushed themoon, your delight color orange for themoon, delight Warm P6:Warm CrushedMoon Orange P6: Crushed Moon WhenOrange Crushed you crushof orange night.Whenof orange night.Crushed moon, you light crush of orangefor night. orange moon, light P6.mp3 for P6.mp3P6.mp3 P6.mp3 orangeorange moon moon orangeRobertRobert moonMurray[58] Murray RobertSmith Smith Murray[58] Hold Smith your colorour sight.Hold for the your delight colorour sight. for the delight Warm P6:Warm CrushedMoon Orange P6: Crushed Moon WhenOrange When you you crushourof crushnight. sight.When orange orange you light lightcrushourof fornight. sight. for orange lightP6.mp3 for P6.mp3P6.mp3 P6.mp3 orange moon orangeRobert moonMurray[58] [58 RobertSmith] Murray[58] Hold SmithCrushed your colorof orangenight.Hold for Crushedthe moon,your delight colorof orangenight. for the moon, delight Warm P6:Warm CrushedMoon OrangeP6: Crushed MoonWhen Orange you crushourour Whensight. orange sight. you light crushour for sight. orange lightP6.mp3 for P6.mp3P6.mp3 P6.mp3 orange moon orangeRobertP7: Beneath moonMurray[58] the RobertSmith P7Dark: Beneath Murray[58] theHold Smith Dark your ofcolor night.Hold for the your delight ofcolor night. for the delight orangeWarm moon orangeRobertP7: WarmBeneath moonMurrayMoon the RobertSmith P7Dark: Beneath MurrayMoon theWhen I Smiththink Dark you of crushusour beneath Whensight. orangeI think you the oflight trees,crushusour forbeneath sight. orange the light trees,P6.mp3 for P6.mp3P6.mp3 P6.mp3 Cool dark CoolGreen dark[58] Grove Green[58] Grove WhenI think you of crushusof beneath night.When Iorange think you the oflight trees,crushusof beneathfor night. orange the light trees, for orange moon orangeRobertP7: BeneathP7 moonMurray: Beneath the RobertSmith P7Dark the: Beneath Dark Murray theBeneath Smith Dark theour dark sight.Beneath green grove.theour dark sight. green grove.P7.mp3P6.mp3 P7.mp3P6.mp3P7.mp3P6.mp3 P7.mp3P6.mp3 CoolGreen dark MeganCoolGreenGreen dark [58]Cooper Grove [59]Megan Green [58]Cooper Grove [59] I think of us beneath the trees, P7.mp3 P7.mp3P7.mp3 P7.mp3 P7: BeneathGreen the P7 GroveDark: Beneath theWhenBeneathI thinkDark you of the usourcrush darkbeneath sight.WhenBeneath Iorange thinkgreen youthe of light grove.the trees,usourcrush dark beneathfor sight. orange green the lightgrove. trees, for CoolCoolGreen dark dark Green MeganCoolGreenGreen dark Cooper [58] Grove [59]Megan Green Cooper[58] Grove [59] Beneath the dark green grove. (or Cool P7(or: Beneath CoolMegan the Cooper P7Dark: Beneath[ 59] theIBeneath think Dark of the usour beneathdark sight.IBeneath think green the of grove. the trees,usour beneathdark sight. green the grove. trees,P7.mp3 P7.mp3P7.mp3 P7.mp3 (orCool(or CoolGreen Cool dark brown) MeganCool(orGreenGreen Cool dark Cooper Grove [59]Megan Green Cooper Grove [59] brown) P7P8:brown) Beneath: Color thePoems; P7DarkP8: Beneath : ColorBrown thePoems;IBeneath think inDark the of colorthe usBrown darkbeneath ofIBeneath naturethink ingreen the the of itself. colorgrove. the trees,us darkbeneath ofBrown nature green the itself.grove. trees,P7.mp3 Brown P7.mp3P7.mp3 P7.mp3 ElectronicsCool(orGreen treeCool dark 2021 , 10, Electronicsx FORMeganCool(orGreenGreen PEER Cool dark 2021 P8Cooper REVIEW : Grove, Color 10, x FOR[59]Megan Poems;Green PEER Cooper REVIEW GroveI think [59]Brown of inus the beneath colorI think of the natureof trees,us beneath itself. the trees,P7.mp3 P7.mp3P7.mp3 P7.mp3 Coolbrown)tree dark P7CoolP8brown): GreenBeneathtree: Colordark Grove Poems; the P7Dark P8: GreenBeneath: Color Brown Grove Poems; theBeneath in Dark the colortheBrown dark ofBeneath nature ingreen the itself. colorgrove.the dark ofBrown nature green itself.grove. Brown (orGreen Cool Megan(orGreen CoolBrown Cooper Brown [59]Megan Brown Cooperis IBrown thethink [59] color isof the usof color beneathanis Ioakwood the ofthink an color the oakwood of trees, usshelf.of beneathan shelf. oakwood the trees, P7.mp3shelf.P8.mp3 P7.mp3P8.mp3P7.mp3P8.mp3 P7.mp3P8.mp3 Electronicsbrown)tree 2021 , 10, Electronicsx FORP8brown) treePEER: Color 2021 REVIEW , 10Poems;, x FOR P8 PEER: ColorBrown REVIEW Poems;Beneath in the colortheBrown dark ofBeneath nature greenin the itself.grove.colorthe dark ofBrown nature green itself.grove. Brown (orCoolGreen Cool dark Megan(orCoolAlexGreenGreen Cool dark BrownSafford AlexCooper Grove Safford [60] [59]Megan Alex [ Green60] BrownSafford Cooper GroveisBrown the[60] [59] Brown color is the isofthe trunksanisBrown trunksoakwood the ofcolor ofisthe thethe shelf.oftrees trees trunksan oakwood of the shelf.treesP8.mp3 P8.mp3P8.mp3 P8.mp3 brown)tree P8brown)tree: Color Poems;P8 : ColorBrown Poems;Beneath in the color theBrown dark of Beneathnature in green the itself.color grove.the dark ofBrown nature green itself. grove.P7.mp3 Brown P7.mp3P7.mp3 P7.mp3 (orGreen Cool Megan(orAlexGreen Cool SaffordBrown Cooper [60] [59]MeganAlex SaffordBrown CooperisBrown the[60] [59] color is the of trunks anisBrown oakwood the of color isthe the treesshelf.of trunks an oakwood of the treesshelf.P8.mp3 P8.mp3P8.mp3 P8.mp3 Electronicsbrown)tree 2021 , 10, Electronicsx FORP8brown) PEERtree: Color 2021 REVIEW , 10Poems;Table, x FOR P8 7. PEER :Poetry ColorBrown REVIEW for Poems;Table representing in the7. Poetry colorBrown colors for of representingnature in in the Henry itself.color Matisse’s colors ofBrown nature in “Dance”. Henry itself. Matisse’s Brown “Dance”. brown)(or Cool brown)(orAlex Cool BrownSafford [60]Alex BrownSaffordisBrown the [60] color is the of trunksanisBrown oakwood the ofcolor isthe the shelf.oftrees trunksan oakwood of the shelf.treesP8.mp3 P8.mp3P8.mp3 P8.mp3 Cooltree green P8tree: ColorP9Brown : Green: Poems; TheP8 : Color ColorBrownBrown Poems;is My thein favoritethe color colorBrown of colors anofis oakwoodnature thein are the color green itself.color shelf.of and anofBrown brown. oakwoodnature itself. shelf. Brown ElectronicsCoolbrown) green 2021 , 10 , Electronicsx FORP9CoolAlexbrown): PEERGreen: green 2021Safford REVIEW , The10 , x [60]FORColorP9 Alex: PEERGreen: Safford REVIEWMy The favoriteBrown [60]Color is colors theMy trunks are favoriteBrown green of isthe colors andthe trees trunksbrown. are green of the and treesP8.mp3 brown. P8.mp3P8.mp3 P8.mp3 tree P8tree: ColorBrown TablePoems; 7.P8 Poetry: ColorBrownBrown for TablePoems;is representing the in Part7.thecolor Poetry colorof ofBrown Poem colors foran ofis representingoakwood naturethe inExcerptinPart theHenrycolor itself. colorof shelf.of from Matisse’sPoem colors an ofBrown oakwood thenature Excerptin “Dance”.Henry itself. shelf. fromMatisse’s BrownModulated the “Dance”. Voice Modulated Voice Cooltown greenColor P9CoolAlex: Green: greenColor SaffordBri The Edwards [60] PoetColorP9Alex: Green: [61 ]SaffordMy The favoriteBrownGreen-leafed [60] PoetColor is colors theMy trunks trees are favoriteBrown polka-dotgreen of isthe colors andthe trees our trunksbrown. are town. green of the and treesP8.mp3P9.mp3 brown. P8.mp3P9.mp3P8.mp3P9.mp3 P8.mp3P9.mp3 towntree AlexBritowntree Edwards SaffordBrown [60][61]AlexBri Edwards SaffordBrownGreen-leafedBrownis [60]the[61] color is the trees ofGreen-leafed Originaltrunks an polka-dotBrownis oakwood the of colorPoem isthe the treesour treesshelf.of Originaltrunks antown. polka-dot oakwood of Poem the our treesshelf. town. Cooltown green P9CoolBri:town Green: Edwardsgreen TheTable [61]ColorP9 7.Bri: PoetryGreen: Edwards MyGreen-leafed for TheTable favoriterepresenting [61]Color 7. Poetry colors treesMy Green-leafedcolors for are polka-dot favoriterepresenting ingreen Henry colors and treesour Matisse’s colors brown.town. arepolka-dot ingreen “Dance”.Henry and our Matisse’sP9.mp3P8.mp3 Word brown.town.Modulated “Dance”. P9.mp3P8.mp3Voice WordP9.mp3P8.mp3 PhraseModulated VoiceP9.mp3P8.mp3 Phrase Part of Poem ExcerptPart of from Poem the Excerpt from the Cooltown greenColor P9CoolBriAlex: townGreen: greenEdwardsColor Safford The PoetColor[61][60]P9BriAlex: Green: Edwards SaffordMyGreen-leafed The favoriteBrown PoetColor[61][60] is colors thetreesMyGreen-leafed trunks are favoritepolka-dotBrown green of isthecolors and thetreesour trees brown.trunks town.are polka-dot green of the and our treesP9.mp3 brown. town. P9.mp3P9.mp3 P9.mp3 P10: My ImaginationP10: My of ImaginationCool,Original ofdeep, fathomless PoemCool, Original deep, fathomless Poem Word Word Phrase Phrase Cooltown green P9CoolBri: townGreen: Edwardsgreen Table The Table 7.Color[61] P9Poetry 7.Bri: Green:Poetry Edwards forMyGreen-leafed for representingTheTable favorite representing Color[61] Part7. Poetry colors of colorstrees MyPoem Green-leafedcolors for are favoritepolka-dot inrepresenting Henry ExcerptgreeninPart Henry colors Matisse’sandof treesour from Matisse’sPoem colors brown. town.are polka-dot the “Dance”. Excerptgreenin “Dance”. Henry and our from Matisse’sP9.mp3 brown. town.Modulated the “Dance”. P9.mp3Voice P9.mp3Modulated VoiceP9.mp3 Color Color Poet a Poet a a perfect shade aof perfect blue shade of blue Cooltown green P9CoolBri: townGreen: Edwardsgreen The [61]ColorP9Bri: Green: EdwardsMyGreen-leafed The favorite [61]Color colors treesMyGreen-leafedOriginal are favoritepolka-dot green Poem colors and treesour Original brown. town.are polka-dot green Poem and ourP9.mp3 Word brown. town. P9.mp3WordP9.mp3 Phrase P9.mp3 Phrase Blue sky BlueP10: skyMy ImaginationP10: My of ImaginationCool, ofdeep, fathomlessCool, deep, fathomlessP10.mp3 P10.mp3P10.mp3 P10.mp3 Cooltown green P9CoolBri:town Green: Edwards green The Purple [61] ColorP9Bri: SkyGreen: Edwards MyGreen-leafed ThePurple favorite [61] ColorPartPart Skythe colorsof oftrees green Poem MyGreen-leafed arepolka-dot favorite of ExcerptExcerpt green SicilianPartthe colors of fromandtreesour fromgreen Poemolives, brown.town. the arepolka-dot the of Excerpt green Sicilian andour from P9.mp3olives, brown.town.Modulated the Voice P9.mp3VoiceP9.mp3Modulated VoiceP9.mp3 Color Color Poet Poeta Poeta a perfect shade aof perfect blue shade of blue townColor BritownP10 Edwards: My Imagination [61]Bri P10 Edwards: Green-leafedMy of Imagination [61] Cool, treesOriginalGreen-leafed ofdeep, polka-dot fathomless PoemCool, treesour deep,town. polka-dot fathomless ourP9.mp3 town. P9.mp3P9.mp3 P9.mp3 Blue sky BluePremila sky Patel [62]Premila Patelsunsets [62]Original orange Poem dazzle,sunsets Original orange Poem dazzle, P10.mp3WordWord P10.mp3 PhraseWordP10.mp3 Phrase P10.mp3 Phrase Purple a Sky Purple a Skythea perfectgreen of shade Sicilianthe aof perfectgreen blueolives, of shade Sicilian of blueolives, P10: My Imagination of Cool, deep, fathomless Blue sky BlueP10Premila: skyMy Imagination PatelP10 [62]Premila: My of Imagination PatelCool,sunsets [62] ofdeep, orange fathomless Cool,dazzle,sunsets deep, orange fathomless dazzle,P10.mp3 P10.mp3P10.mp3 P10.mp3 Purplea Sky Purple Green Skythea green perfect the optimist of shade SicilianGreenthe of and bluegreen olives,the the optimist ofeye Sicilian and olives, the eye BlueCool sky Cool a a a perfect shade aof perfect blue shade of blue Blue sky BluePremila skyPurpleP11 : PatelGreen Sky [62] Premila P11: PatelGreenthesunsets [62] green oforangeopener Sicilian dazzle,sunsets olives, orangeopener dazzle, Dark Green Dark GreenPurple Sky PurpleGreen Skythe green the optimist of SicilianGreenthe and green olives,the the optimist ofeye Sicilian and olives,P10.mp3 the eye P10.mp3P10.mp3 P10.mp3 Cool CoolPremilaSyed Ali Patel Sagar [62] [63]Syed Ali SagarGreensunsets [63] means orange cleanGreen dazzle, and means hope clean andP11.mp3 hope P11.mp3P11.mp3 P11.mp3 Grass GrassPremilaP11 : PatelGreen [62] Premila P11: PatelGreensunsets [62] orangeopener dazzle,sunsets orangeopener dazzle, DarkCool Green Dark Green GreenGreen theGlory, optimistthe optimistBlossom andGreen theand Glory,and eyethe the Life opener optimistBlossom eye and and the Life eye Cool CoolSyedP11 Ali: Green Sagar [63]Syed Ali SagarGreen [63] means cleanGreen and means hope clean andP11.mp3 hope P11.mp3P11.mp3 P11.mp3 Dark Green P11: Green P11: GreenGreen meansopener clean and hopeopener Grass GrassSyed Ali Sagar [63] Dark Green Dark Green GreenGlory, the optimistBlossomGreen andGlory,and the theLife optimistBlossom eye andandP11.mp3 theLife eye P11.mp3P11.mp3 P11.mp3 GrassCool CoolSyed Ali Sagar [63]Syed Ali SagarGreenGlory, [63]Orange means Blossom is clean theGreen and sunset andOrange Life means hope is clean the sunset and hope Grass GrassP11 : Green P11: Green opener opener P12: Orange P12: OrangeGlory,Orangeits fire, Blossom isfire the is sunset Glory,andorangeits Life fire, Blossom fire is andorange Life Dark Green Dark Green Orange is the sunsetOrange is the sunsetP11.mp3 P11.mp3P11.mp3 P11.mp3 Syed(ColorP12 Ali: Orange Poem) Sagar [63][64]Syed(Color Ali Poem) SagarGreenalso [63][64]its means ﬁre,the most ﬁre clean isGreen orangevividalso and means colorthe hope most clean vivid and P12.mp3color hope P12.mp3P12.mp3 P12.mp3 Grass GrassP12 : Orange P12: Orangeits fire, fire is orangeits fire, fire is orange (Color Poem) [64] Glory,alsoOrange the Blossom most is the vivid Glory,andsunsetOrange color Life Blossom is the andsunset Life God dared to make.God dared to make.P12.mp3 P12.mp3P12.mp3 P12.mp3 (Color Poem) [64](Color Poem)also [64]God the dared most to vivid make.also colorthe most vivid color WarmWarm orange orange Body Warm orangeP12: Orange P12: Orangeits Thefire, skyfire is greenorangeits Thefire, skyfire is greenorange (Color Poem) [64](Color Poem)also [64]OrangeGod Thethe dared skymost is isthe greenvividto alsosunset make.OrangeGod colorthe dared most is the vividto sunset make. P12.mp3color P12.mp3P12.mp3 P12.mp3 Body Body andand the the grass grass is is blue andblue the grass is blue Warm orange WarmP13 orange:P12 Orange: Orange is Blue P12: Orangeits Thefire, skyfire isis greenorangeits Thefire, skyfire isis greenorange P13: Orange is BlueP13: Orange is GodBluethethe dared sky sky is is greento green make.Godthe dared sky is to green make. Body BodySummer(Color Poem) Song [65 [64]](Color Poem)also [64] the most vividalso colorthe most vivid P12.mp3color P12.mp3P12.mp3 P12.mp3 Warm orange Warm orange andand theThe the grass sky grass is bluegreen is andblue blueThe the sky grass is green is blueP13.mp3 P13.mp3P13.mp3 P13.mp3 Summer Song [65]Summer Song God[65]and thedared grass to make.isGod andblue thedared grass to make.is blue Body BodyP13: Orange is BlueP13: Orangeis is orange Bluethe and sky orange is green is bluethe sky is green and bluethe grass is orange is andblue bluethe grass is orange is blue Warm orange WarmSummer orange Song [65]Summer Song [65]andThe the sky grass is green is andblueThe the sky grass is green is blueP13.mp3 P13.mp3P13.mp3 P13.mp3 P13: Orange is BlueP13: Orange is Blueandthe orangesky is green is blueandthe orangesky is green is blue Body Body Summer Song [65]Summer Song and[65]blue the grassis orange is andblue blue the grassis orange is blueP13.mp3 P13.mp3P13.mp3 P13.mp3 P13: Orange is BlueP13: Orange is Blueandtheblue orangesky is isorange green is blueand the blue orangesky is isorange green is blue 3.6. Implicit Association3.6. Implicit Test Associationto Find a Solution Test to to Find the aProblem Solution P13.mp3of toColorPoetry the Problem P13.mp3ofP13.mp3 ColorPoetry P13.mp3 Summer Song [65]Summer Song [65]andand the orange grass is is blue andblueand the orange grass is is blue blue 3.6. ImplicitThe purpose Association3.6. of Implicit Thethis Test purposetest Associationtoblue isFind to is findofa orange Solutionthis Testa solutitest to blue to isFindon theto is to findaProblem orange theSolution a algorithm soluti of toColorPoetryon the to Problemdescribed the algorithm of inColorPoetry Section described 3.2 in Section 3.2 for the instance forof Table the instance 6.and To do orange of so, Table an is implicit 6.blueand To do orange association so, an is implicit blue test associationwas performed test wasto identify performed to identify 3.6. ImplicitThe purpose Association3.6. of Implicit Thethis Test purposetest Associationto isFind to findofa Solution this Testa solutitest to toisFindon theto to findaProblem Solutionthe a algorithm soluti of toColorPoetryon the to Problemdescribed the algorithm of ColorPoetryin Section described 3.2 in Section 3.2 the intimacy of the colorintimacy stimulus of the in color the poemsstimulus (Table in the 6) poems and the (Table one in 6) the and artwork the one (“The in the artwork (“The for the instance forof Table the instance 6. To do of so, Table an implicit 6. To do association so, an implicit test associationwas performed test wasto identify performed to identify starryThe night” purpose bystarry Van of Thethis Gogh)night” purposetest is throughby to Van findof this Gogh) thea solutitest intervention isthroughon to tofind the thea ofalgorithm soluti interventiona semanticon to described the differential ofalgorithm a semantic in Section adjectivedescribed differential 3.2 in Section adjective 3.2 3.6.the Implicitintimacy Association of 3.6.the Implicitcolorintimacy Test stimulus Associationto of Find the ina color Solution the Test poemsstimulus to to Find the (Table inaProblem Solution the 6) poems and of toColorPoetry the the (Table one Problem in 6) the and of artwork ColorPoetry the one (“The in the artwork (“The forantonym the instance in Tables forofantonym Table the 4 and instance 6. in 5.To TablesFifteen do of so, Table 4anstudents and implicit 6. 5.To Fifteen dowere association so, recruited anstudents implicit test aswere associationwas participants recruited performed test ofas was totheparticipants identify experi-performed of tothe identify experi- starry night” bystarry Van Gogh)night” throughby Van Gogh)the intervention through the of interventiona semantic differential of a semantic adjective differential adjective thement. intimacyThe The purpose gender of thement. of ofcolorintimacy The thisthe The stimulusparticipantspurposetest gender ofis theto findofin ofcolor thisthethe was a poemssolutistimulusparticipantstest 5 ma ison les to(Table to findinand the the was 10a6) algorithm poemssoluti andfemales, 5 ma theonles (Table toone andanddescribed the in 106)the algorithmthe andfemales, average artwork in the Section one anddescribed age (“The in the was3.2 the average artwork in Section age (“The was3.2 antonym in Tablesantonym 4 and in5. TablesFifteen 4students and 5. Fifteen were recruited students aswere participants recruited ofas theparticipants experi- of the experi- forstarry22.3 the years night”instance (minimum by forstarryof 22.3Van Table the years Gogh) night”instance20 6. years To(minimum through bydo of old, Vanso, Table anmaximum Gogh)the 20 implicit 6. interventionyears To through do associationold,27 so, years anmaximum the ofimplicit old).interventiona semantictest The associationwas27 yearsnumber performed differential of old).a semantic testof Theparticipants wasto adjective identify number performed differential of participants to adjective identify ment. The genderment. of the The participants gender of the was participants 5 males and was 10 females, 5 males andand 10the females, average and age thewas average age was theantonymwith intimacy music in Tables andof theantonymwith literature colorintimacy4 andmusic stimulus in5. experience Tables Fifteenandof the literature in color 4students theand is poemsstimulus 5.9 experienceanFifteen wered (Table 5, in recruitedand students the 6) is thepoems and9 anaverage as werethed (Table participants5, one recruitedand number in 6) thethe and averageartwork ofas theof the participantsyears one experi- number(“The in they the artwork of of the years experi- (“The they 22.3 years (minimum22.3 years 20 years (minimum old, maximum 20 years old,27 years maximum old). The 27 yearsnumber old). of Theparticipants number of participants starryment.have experiencedThenight” gender bystarryment. haveVan of is the 5.5Gogh)experiencedThenight” participants years gender throughby and Van of is 3.5 the was 5.5Gogh)the years participants years 5intervention ma through (Figure lesand and 3.5 was3). the10 ofyears While females,5interventiona masemantic (Figureles participating andand differential3). 10 theof Whilefemales, aaverage semantic in participating the adjectiveand age experi- differential thewas average in the adjective age experi- was with music andwith literature music experience and literature is 9 experienceand 5, and isthe 9 anaveraged 5, and number the average of years number they of years they antonym22.3ment, years repeated in (minimum Tablesantonym22.3 ment,auditory 4 years and 20repeated inyears 5.(minimumstimulation TablesFifteen old, auditory 4studentsmaximum and 20 may years 5.stimulation Fifteen causewere old,27 yearsrecruited sidestudentsmaximum may effectsold). causeaswere The 27participantssuch yearsnumberrecruitedside as effectsold).headaches, of ofas Theparticipants theparticipantssuch numberexperi- andas headaches, if of of participants the experi- and if have experiencedhave is 5.5experienced years and is 3.5 5.5 years years (Figure and 3.5 3). years While (Figure participating 3). While in participating the experi- in the experiment.withphysical music The or gender andmentalment.with physicalliterature of discomfort themusic The participants or gender experience andmental isliterature offelt, discomfortthe was isparticipants participants 95 experiencemaandles is5, andfelt,and are was 10 isparticipants theinformed females, 95 maanaveragedles 5, and andinand are numberadvance 10the theinformed females, average average of that years andin agethey numberadvance thetheywas can average of that years agethey theywas can ment, repeated ment,auditory repeated stimulation auditory may stimulation cause side may effects cause such side as effectsheadaches, such andas headaches, if and if 22.3haverequest years experienced to (minimumstop 22.3havetherequest is experiment years 5.5experienced 20 yearsto years (minimumstop and old,theat is any3.5 experiment maximum5.5 20 yearstime. years years (Figure and old,27at anyyears3.5 maximum 3). yearstime. Whileold). (Figure The participating27 yearsnumber 3). Whileold). of in The participantsparticipating the number experi- of in participants the experi- physical or mentalphysical discomfort or mental is felt, discomfort participants is felt, are participants informed in are advance informed that in they advance can that they can withment, musicThe repeated participants and withment, auditoryliterature musicThe firstrepeated participantsstimulation completedexperience and auditoryliterature afirstmayis training 9stimulation completedexperiencean caused 5,session andside a mayis the training effectsthat 9 an averagecause lastedd such 5,session andside approximatelynumber as the effectsthatheadaches, average lastedof suchyears an approximatelynumber andas hour,they headaches, if of years an and hour,they if request to stop therequest experiment to stop theat any experiment time. at any time. havephysicalfollowed experienced or by mental anhavephysical followedhour is discomfort 5.5experienced of yearsevaluation.or by mental an and is hour felt,is 3.5discomfort 5.5During of participantsyears yearsevaluation. th(Figure and eis training felt, 3.5are During3). participants yearsinformed While session, th(Figure eparticipating training inthe are advance 3). participants informed While session, inthat participating the inthe theyfamiliar- experi-advance participants can inthat the theyfamiliar- experi- can The participantsThe first participants completed afirst training completed session a training that lasted session approximately that lasted anapproximately hour, an hour, ment,requestized themselves repeated to stop thement,requestizedauditory with experiment themselves repeatedthe to stimulation stopconcept theatauditory withany experimentbehind time.maythe stimulation concept thecause atsound any sidebehind time.(modulatedmay effects thecause soundsuch sidevoices) as (modulated effectsheadaches, and suchpoem voices) andas to headaches, beif and poem and to beif followed by an followedhour of evaluation. by an hour During of evaluation. the training During session, the training the participants session, the familiar- participants familiar- physicalusedThe to expressorparticipants mentalphysicalused various discomfortThe firstto expressor participantscolor completed mental isappearance variousfelt, discomfort afirstparticipants training color completed dime is appearancesession nsionsfelt, are aparticipants informedtraining that through dimelasted session nsionsintutorial approximatelyare advance informed that through materials, lasted that in tutorial they anapproximately advance Tableshour, can materials, that theyan Tableshour, can ized themselvesized with themselves the concept with behind the conceptthe sound behind (modulated the sound voices) (modulated and poem voices) to be and poem to be requestfollowed1–7, and to byFigures stop an therequestfollowedhour1–7, 1 experimentand andof to evaluation.2. byFigures stopThe an thesoundathour any1 experiment andDuring of time.source evaluation.2. The th ane soundattrainingd anyuser During time.source testimonial session, th ane trainingd the user questions participants testimonial session, to bethe familiar- questionsused participants in to be familiar- used in used to expressused various to express color appearance various color dime appearancensions through dime nsionstutorial through materials, tutorial Tables materials, Tables izedthe experimentThethemselves participantsized thewerewith experimentThethemselves firstdistributedthe participants completedconcept werewith just behind afirst distributedthebefore training completedconceptthe the sound session experiment just behind a (modulatedbefore training that the lasted thebegan. sound session experiment voices)approximately After (modulated that andorientation, lasted began. poem anvoices)approximately After hour,to par- be andorientation, poem an hour,to par- be 1–7, and Figures1–7, 1 and and 2. Figures The sound 1 and source 2. The an soundd user source testimonial and user questions testimonial to be questionsused in to be used in followedusedticipants to express bywere an followedpresentedusedhour ticipantsvarious toof evaluation.express color by werewith an appearance presentedhour soundvarious During of and evaluation. color with dimevoiceth eappearance soundtrainingnsions clips During andshown through session, dimevoiceth ein training nsions tutorialtheTablesclips participants shown through2,session, materials, 3, 6 in and tutorialtheTables familiar- 7.Tables participants After 2, materials, 3, 6 and familiar- 7.Tables After the experiment thewere experiment distributed were just distributedbefore the experiment just before thebegan. experiment After orientation, began. After par- orientation, par- ized1–7,that, andthemselves the Figuresparticipantsized1–7, that, with1 and andthemselves the evaluatedthe 2. Figuresparticipants Theconcept sound withthe1 behindand impl sourceevaluatedthe 2.icit Theconceptthe associationan soundd the user behind impl source(modulated testimonial andicit the associationanuser soundd user experience.voices)questions (modulated testimonial and and userto poem be experience.voices) questionsused to bein and to poem be used to bein ticipants were presentedticipants werewith presentedsound and with voice sound clips andshown voice in Tablesclips shown 2, 3, 6 in and Tables 7. After 2, 3, 6 and 7. After usedthe experiment to express usedthe werevarious experiment todistributed express color appearance werevarious just distributedbefore color dime the appearance experimentnsions just before through dime thebegan. experimentnsionstutorial After through materials, orientation, began. tutorial AfterTables par- materials, orientation, Tables par- that, the participantsthat, the evaluated participants the impl evaluatedicit association the impl andicit associationuser experience. and user experience. 1–7,ticipants and Figureswere presented1–7,ticipants 1 and and 2. Figures were withThe soundpresentedsound 1 and sourceand 2. withThe voice an soundsoundd clips user sourceand showntestimonial voice an ind Tablesclips user questions showntestimonial 2, 3, 6 toin and beTables questions7.used After 2, in 3, 6 to and be 7.used After in thethat, experiment the participants thethat,were experiment thedistributedevaluated participants werethe just impl distributedevaluatedbeforeicit associationthe theexperiment just impl before andicit associationthebegan.user experiment experience. After and orientation, began.user experience. After par- orientation, participants were presentedticipants werewith presentedsound and with voice sound clips andshown voice in Tablesclips shown 2, 3, 6 in and Tables 7. After 2, 3, 6 and 7. After that, the participantsthat, the evaluated participants the impl evaluatedicit association the impl andicit associationuser experience. and user experience.

Electronics 2021, 10, 1064 14 of 24

3.6. Implicit Association Test to Find a Solution to the Problem of ColorPoetry The purpose of this test is to ﬁnd a solution to the algorithm described in Section 3.2 for the instance of Table6. To do so, an implicit association test was performed to identify the intimacy of the color stimulus in the poems (Table6) and the one in the artwork (“The starry night” by Van Gogh) through the intervention of a semantic differential adjective antonym in Tables4 and5. Fifteen students were recruited as participants of the experiment. The gender of the participants was 5 males and 10 females, and the average age was 22.3 years (minimum 20 years old, maximum 27 years old). The number of participants with music and literature experience is 9 and 5, and the average number of years they have experienced is 5.5 years and 3.5 years (Figure3). While participating in the experiment, repeated auditory stimulation may cause side effects such as headaches, and if physical Electronics 2021, 10, x FOR PEER REVIEWor mental discomfort is felt, participants are informed in advance that they can request to

stop the experiment at any time.

FigureFigure 3. 3.Demographic Demographic data data on on participants’ participants’ experience experience in in music music and and literature. literature.

TheOsgood participants [66] simplified first completed the semantic a training space session of relative that lasted adjectives approximately into three an aspects: hour, followedevaluation, by an efficacy, hour of and evaluation. activity. During The pair the of training semantic session, derivatives the participants that were familiarized adopted in themselvesthis experiment, with the including concept some behind adjectives the sound in [66], (modulated are the pair voices) of adjectives and poem (Tables to be used 4 and to5) express people variousare familiar color with appearance for the light, dimensions dark, warm, through and tutorialcold color materials, of each hue Tables collected1–7, andthrough Figures literature1 and2. research The sound [24]. sourceIn our andexperi userments, testimonial participants questions are asked to beto usedexperience in themultiple experiment imagery were sensations distributed (see just the beforedefinition the of experiment imagery in began. Section After 2.1) orientation,like not only participantssight, but also, were sound, presented touch, with and sound smell and from voice the clipsartwork shown (Van in Gogh’s Tables2 “The,3,6 and starry7. After night”) that,and thepoems participants in Table evaluated6, respectively. the implicit Imagery association with those and four user sensations experience. is used for partic- ipantsOsgood to appreciate [66] simplified the colors the in semantic artwork spaceand poems. of relative The sound adjectives is produced into three by aspects:means of evaluation,voice narration efficacy, of andpoems activity. adjusting The pairthe ofintensity, semantic pitch, derivatives and reverberation that were adopted to distinguish in this experiment,warm/cool/vivid/light/dark including some adjectivesdimensions in of [66 colors], are the(see pair Section of adjectives 3.4) so that(Tables the4 sound and5) is peoplematched are with familiar the characteristics with for the light, of colors dark, in warm, artwork. and cold color of each hue collected throughTest literature participants research experienced [24]. In ourtwo experiments, poem stimuli participants in Table 6 expressing are asked each to experience color stim- multipleulus in imagerythe artwork sensations and the (see colors the definitiondescribed of in imagery the poems in Section they thought 2.1) like were not only associated sight, butwith also, that sound, stimulus. touch, For and each smell artwork from or the poem artwork stimulus, (Van Gogh’s there were “The provided starry night”) to the and par- poemsticipants in Table a total6, respectively.of 14 concept Imagery adjectives with pairs those in Tables four sensations 4 and 5. is used for participants to appreciate the colors in artwork and poems. The sound is produced by means of Table 8 illustrates an implicit association test questionnaire for the case of color stim- voice narration of poems adjusting the intensity, pitch, and reverberation to distinguish uli “Blue sky and night” in artwork associated with Poem 1 and Poem 2. The following warm/cool/vivid/light/dark dimensions of colors (see Section 3.4) so that the sound is test guideline is given to the participants: matched with the characteristics of colors in artwork. “Based on your experience on each color in the artwork and two poems expressing Test participants experienced two poem stimuli in Table6 expressing each color the color today, check the box that reflects your immediate response to each adjective pair stimulus in the artwork and the colors described in the poems they thought were associated in the given table. Do not think too long to select a score among (2, 1, 0, −1, −2), and write with that stimulus. For each artwork or poem stimulus, there were provided to the it in each empty box. Make sure you respond to every adjective pair. Each of these 14 pairs participants a total of 14 concept adjectives pairs in Tables4 and5. of semantic differential adjectives, a visual feeling that is in context with the various sensations conveyed by the colors of art works or poem 1 and poem 2 is given 2 points for the closest one to the former, 1 point for the closer one to the former, −1 point for the closer to the latter, −2 points for the closest to the latter. Otherwise, if you don’t know how to respond, simply score 0.” In this way, the similarity of Art–Poem Association with respect to colors denoted as S (A, P) for each of the two stimuli A and P is measured. Test results are shown in Figure 4. Then, the set of poems P having sigmax S (A, P) (as defined in Section 3.2) is selected to represent the color C in the artwork.

Electronics 2021, 10, 1064 15 of 24

Table8 illustrates an implicit association test questionnaire for the case of color stimuli “Blue sky and night” in artwork associated with Poem 1 and Poem 2. The following test guideline is given to the participants:

Table 8. An implicit association test questions for the case of color stimuli “Blue sky and night” in artwork associated with Poem 1 and Poem 2.

(a) The Semantic Differential Pair Relevant to Warmer of Cooler Color of Each Hue No The Semantic Differential Pair S(A) S(P1) S(P2) 1 Strong~Weak (sound) - - - 2 Near~Far (thermal-tactile) - - - 3 Warm~Cool (vision) - - - (b) The Semantic Differential Pair Relevant to Lighter/Darker Color of Each Hue 1 Soft~Hard (haptic) - - - Round (Bouba)~Pointed (Kiki) 2 --- (haptic) 3 Smooth~Rough (haptic) - - - 4 Light~Heavy (haptic) - - - 5 Light~Heavy (sound) - - - 6 High~low (sound) - - - 7 Aroused~Calm (vision) - - - 8 Light~Dark (vision) - - - 9 Pleasant~Unpleasant (scent) - - - 10 Weak~Strong (scent) - - - 11 Floral~Woody (scent) - - -

“Based on your experience on each color in the artwork and two poems expressing the color today, check the box that reﬂects your immediate response to each adjective pair in the given table. Do not think too long to select a score among (2, 1, 0, −1, −2), and write it in each empty box. Make sure you respond to every adjective pair. Each of these 14 pairs of semantic differential adjectives, a visual feeling that is in context with the various sensations conveyed by the colors of art works or poem 1 and poem 2 is given 2 points for the closest one to the former, 1 point for the closer one to the former, −1 point for the closer to the latter, −2 points for the closest to the latter. Otherwise, if you don’t know how to respond, simply score 0.” In this way, the similarity of Art–Poem Association with respect to colors denoted as S (A, P) for each of the two stimuli A and P is measured. Test results are shown in Figure4. Then, the set of poems P having sigmax S (A, P) (as deﬁned in Section 3.2) is selected to represent the color C in the artwork. We measured cosine similarity between two vectors (like artwork A and poem P). The values of two vectors should be positive. Thus, the score range of (2, 1, 0, −1, −2) was replaced with (4, 3, 2, 1, 0). Since the similarity is measured by the cosine value of the angle formed by two vectors, the cosine similarity gets closer to 1 as the angle is smaller, and closer to 0 as the angle is larger. We obtained the similarity for Blue sky: S (A, P1) = 0.978 and S (A, P2) = 0.901, for Blue wind: S (A, P4) = 0.871 and S (A, P5) = 0.203, and for Green cypress: S (A, P7) = 0.501 and S (A, P8) = 0.569, respectively. According to the analysis based on cosine similarity, the three poems P1, P4, and P8 showed the most consistent dark blue, light blue, and dark green directivity respectively, among the poems used in the experiment settings. The poem depicting Van Gogh’s Starry Night, created through the artwork–poetry matching process presented in this paper, is shown in Table9. ElectronicsElectronics 20212021,, 1010,, x 1064 FOR PEER REVIEW 16 of 24

FigureFigure 4. 4. ComparisonComparison of of response response to to semantic semantic differential differential similarity similarity between between artwork artwork and and two two poems poems with with respect respect to to variousvarious sensations sensations (showing (showing graphical graphical re representationpresentation to visualize the similari similarityty between the artwork and two poems).

Electronics 2021, 10, 1064 17 of 24 Electronics 2021, 10, x FOR PEER REVIEW

Table 9. Poetry for representing colors in Van Gogh’s “The Starry Night” (the sound ﬁles developed inTable this 9. research Poetry arefor providedrepresenting separately colors in as Van a Supplemental Gogh’s “The Material). Starry Night”(the sound files developed in this research are provided separately as a supplement material). The Starry Night TheJun Starry Dong Cho,Night Jun Dong2021 Cho, 2021

poem-word.mp3 poem-phrase.mp3

Dark blue sky’s out there, DarkBefore blue daybreak sky’s out is in;there, FallingBefore stars daybreak here and is there, in; FallingNight stars and distanthere and moon, there, NightSleepless and is distant my night; moon, [53] SleeplessA yellow is my starnight; [53] A yellowIn a sky star Twinkling white ones It’s aIn yellow a sky bird MyTwinkling beautiful little white yellow ones bird! It’s a yellow[55] bird My beautifulThe wind little has yellow a blue tail bird! [55] TheThe wind wind also hashas a a white blue tailtail [ 56] The windCrushed also has orange a white moon, tail [56] Hold yourCrushed color fororange the delight moon, of night. WhenHold you your crush color orange for the light delight for our of sight night. [58] When youBrown crush in theorange color light of nature for our itself. sight. [58] Brown is the color of an oakwood shelf. Brown in the color of nature itself. Brown is the trunks of the trees [59] Brown is the color of an oakwood shelf. My favorite colors are green and brown. Green-leafedBrown is the trees trunks polka-dot of the our trees town [59] [61 ] My favorite colors are green and brown. Green-leafed trees polka-dot our town [61] Moreover, we performed a t-test with paired two sample for means. The average of adjective pairs was calculated for each experiment participant, and the difference between Moreover, we performed a t-test with paired two sample for means. The average of P1 and A and the difference between P2 and A was calculated, and the absolute value was adjective pairs was calculated for each experiment participant, and the difference between taken. Respondents whose value is closer to 0 means that the difference between P and A P1 and A and the difference between P2 and A was calculated, and the absolute value was is smaller. The differences between P1 and A and between P2 and A, which is the average taken. Respondents whose value is closer to 0 means that the difference between P and A of the larger one, were verified through a paired t-test. As a result of the verification, the is smaller. The differences between P1 and A and between P2 and A, which is the average verification statistics were t (14) = −1.46, p = 0.17 (two-tail verification), which could not be of the larger one, were verified through a paired t-test. As a result of the verification, the said to be statistically significant. The differences between P4 and A and between P5 and verification statistics were t (14) = −1.46, p = 0.17 (two-tail verification), which could not be A, which is the average of the larger one, were verified through a paired t-test. said Asto be a resultstatistically of the significant. verification, The the di verificationfferences between statistics P4 were and tA (14) and = between−2.06, p P5= 0.03and (two-tailA, which verification), is the average and of therethe larger was a one, significant were verified difference through within a thepaired significance t-test. level of 5%. ThatAs a is, result P4 is of statistically the verification, significant the comparedverification to P5statistics and is similarwere t to(14) A. = The −2.06, differences p = 0.03 between(two-tail P7verification), and A and and between there was P8 and a significant A, which difference was larger within on average, the significance were verified level throughof 5%. That a paired is, P4 is t-test. statistically As a resultsignificant of the compared verification, to P5 the and verification is similar to statistics A. The differ- were tences (14) =between−0.09, p P7= and 0.46 A (two-tail and between verification), P8 and A, which which could wasnot larger be saidon average, to be statistically were veri- significant.fied through According a paired t-test. to these As analyses, a result of we the reject verification, P5 that showed the verification the significant statistics difference were t from(14) = P4. −0.09, Therefore, p = 0.46 finally, (two-tail we verification), could confirm which that thecould set not of poems be said (P1, to be P2, statistically P4, P7, P8) sig- are determinednificant. According to be the to set these of “good” analyses, poems we withreject the P5 maximum that showed significance the significant of the Art–Poemdifference similarityfrom P4. Therefore, score with finally, respect we to eachcould corresponding confirm that the color set stimulus.of poems (P1, P2, P4, P7, P8) are determinedAs a result to be of thethe experimentset of “good” conducted poems towith find the out maximum the color directivitysignificance of eachof the poem, Art– thePoem poem similarity that showed score with the most respect consistent to each colorcorresponding directivity color was stimulus. P1. For P1, the adjectives chosenAs bya result the participants of the experiment as they conducted feel most to suitable find out are the cool color (vision), directivity calm of (vision), each poem, far (vision),the poem dark that (vision), showed low the (sound),most consistent strong (vision),color directivity rough (haptic), was P1. heavy For P1, (sound), the adjectives woody chosen by the participants as they feel most suitable are cool (vision), calm (vision), far

Electronics 2021, 10, x FOR PEER REVIEW

Electronics 2021, 10, 1064 18 of 24

(vision), dark (vision), low (sound), strong (vision), rough (haptic), heavy (sound), woody (scent),(scent), round round (haptic), (haptic), and and weak weak (scent), (scent), in thein the order order of their of their preferences. preferences. Among Among them, them, the adjectivesthe adjectives of the of same the same directivity directivity (positive (positive or negative) or negative) in both in artwork both artwork and P1 withand havingP1 with anhaving intensity an intensity of 0.7 or of more 0.7 or appear more asappear cool as (vision), cool (vision), calm (vision), calm (vision), far (vision), far (vision), and dark and (vision).dark (vision). It can beIt can seen be that seen the that four the adjectives four adjectives matched matched well with well the with color the characteristic color charac- of “blueteristic sky of at “blue night”. sky Therefore,at night”. Therefore, we see that we the see poem that the P1 exhibitspoem P1 a exhibits wider variety a wider of variety color appearanceof color appearance dimensions, dimensions, including including color temperature, color temperature, color depth, color and depth, color and emotion, color emo- as welltion, as as color well lightness.as color lightness. OnOn the the other other hand, hand, pleasant–unpleasant pleasant–unpleasant (scent) (scent) and and soft–hard soft–hard (haptic) (haptic) were were not not such such aa case. case. ParticipantsParticipants reacted reacted “pleasantly” “pleasantly” for for artwork artwork and and unpleasantly unpleasantly for for p1, p1, showing showing the the oppositeopposite directivity. directivity. In In addition, addition, participants participants responded responded with with “hard” “hard” for artwork for artwork and “soft” and for“soft” P1. However,for P1. However, their intensity their intensity of directivity of directivity was too was weak, too aroundweak, around 0.1~0.4. 0.1~0.4. In addition, In ad- “weak–strongdition, “weak–strong (scent)” was(scent)” shown was as shown “0” (random) as “0” (random) in both artwork in both andartwork P1. and P1.

4.4. Usability Usability Test Test and and Result Result InIn thethe usabilityusability evaluation experiment, experiment, the the SUS SUS (System (System Usability Usability Score) Score) test test was was ex- executed.ecuted. During During the the SUS SUS test, test, our our proposed proposed method method “ColorPoetry” “ColorPoetry” in this in thispaper paper was wascom- comparedpared with with “ColorSound” “ColorSound” [26]. [Fifteen26]. Fifteen participants participants who also who attended also attended the implicit the implicit associ- associationation test described test described in Section in Section 3 were3 were asked asked the following the following question: question: “Based “Based on your on your expe- experiencerience on two on twocolor color codes: codes: ColorSound ColorSound [26] and [26 ColorPoetry] and ColorPoetry today, scoretoday each, score of each10 items of 10of itemsSUS test of SUS set that test setreflects that your reﬂects immediate your immediate response response to each statement. to each statement. Don’t think Don’t too thinklong tooabout long each about statement. each statement. Make sure Make you sure respond you respondto every tostatement. every statement. If you strongly If you stronglydisagree disagree then score then 1, and score if you 1, and strongly if you agre stronglye then agree score then 5. Otherwise, score 5. Otherwise, if you don’t if know you don’thow knowto respond, how tosimply respond, score simply 3.” score 3.” AsAs aa result of of the the analysis analysis as as shown shown in Figure in Figure 5, the5, theColorPoetry ColorPoetry received received good good scores scores(74.5%), (74.5%), comparable comparable to ColorSound to ColorSound (74.1%) (74.1%) in the in user the userexperience experience test (Figure test (Figure 5). Col-5) . ColorSoundorSound was was produced produced similarly similarly to to [17] [17 ]for for comparison comparison purposes andand waswas notnot discussed discussed inin detail detail during during the the training training session, session, so so it receivedit received a lowera lower score score than than the the score score received received in thein the previous previous study study [26 ].[26].

FigureFigure 5. 5.System System Usability Usability Scale Scale test test results. results.

Electronics 2021, 10, 1064 19 of 24

Tables 10 and 11 list the participants’ positive and negative feedback after reviewing the two color-coding systems, respectively. Most participants gave both positive and negative feedback. Two participants, C and L, who have more than 10 years of experience in playing musical instruments and 5 years of experiences in writing poems and literature, responded that “The system looks very efficient and very good to handle.” On the contrary, in the negative evaluations other than the positive user feedback in Table 10, participants E and M, who do not have any experience in both music and literature, said, “It will be a little difficult if you don’t understand the proper manual,” and “If there is a simple explanation at first, it seems to be easy to use.” The participant who provided negative feedback gave a low score on SUS statements 4 and 10 since they thought this system will need the support of a technical person and should learn while using this system. Musical sound coding color is suitable for practical use because there are positive evaluations as a result of existing tests [26] when trying to convey colors by using instruments with characteristics that match the colors used in artworks with classical melodies. Representing the various color dimensions (e.g., distinguishing between warm, cool, light, and dark colors) itself into sound is a much more difficult and complex process than just using color names (with poetry, for example). The feeling that comes from a color is very subjective, so deciding which image to express with some effective sound or musical notes only is very difficult. Therefore, the current method of linking the characteristics of a musical instrument to the characteristics of color is objective and easy to access. Just as sighted people learn color names, blind people need to learn not only color names, but also musical codes that correspond to colors [26]. Table 12 shows conflicting user feedback and future works to resolve the conflict.

Table 10. Positive user feedbacks from the System Usability Scale test.

I felt the system’s approach was very creative. I think two systems are easy A to use and have a simple interface. I think two systems are easy to use and have a simple interface. It does not B require special technical knowledge, and I think it will be easy to use regardless of the individual characteristics of the user. I think that it is a system that relies on human senses and feelings rather than D going through thinking. There seems to be no particular difficulty to use the system. I think they are E common in everyday life, such as voices, colors, and poetry. I think that it is a system that relies on human senses and feelings rather than going through thinking. - I don’t need any more knowledge to use the system, but I think that if you have background knowledge about music and poetry provided by the system, it will be of greater help in using the system. It was easier to understand by making the sound different according to the F color. L The system looks very efficient. This system was very good to handle. It seemed simple to use because you can hear and feel it right away. M At first, it was a little awkward to see how it works, but I think it will be useful afterwards Electronics 2021, 10, 1064 20 of 24

Table 11. Negative user feedback from the System Usability Scale test.

It was my first time using it, so I didn’t feel confident about using the system A because I was unfamiliar with it. In the case of sound color, more diverse interpretations are possible depending on the user regardless of the matching between music and color. Therefore, it helps intuitive understanding that can be relative to each user. B However, in the case of poetry, there may be various interpretations of the content for each user, and since the voice matched with poetry excludes such various possibilities, it is judged that it is difficult to feel intuitive from the user’s point of view. C It seems that you should be able to learn how to use it. E It will be a little difficult if you don’t understand the proper manual. Poetry is okay because it is directly connected to color, but the voice with word is unnatural when savoring poetry. G There is no big problem with both when it comes to integration.Poetry seems to be used a lot because the overall system is okay, but music is not. However, in the case of music, people who understand will use it a lot. M If there is a simple explanation at first, it seems to be easy to use. When I first listened without any explanation, it was a little difficult to O evaluate or use.

Table 12. Critical and conﬂicted user feedback and future works.

Conflicted User Feedbacks Conflict Resolution (Future Works) D: The sudden change of voice within a single paragraph seems to be unfamiliar yet. F: Having different voices with words could These conflicted thoughts between participants specifically indicate the meaning to be can be mediated by using both methods of expressed. voice modulation, selectively considering the N: The voice with word units seems to have a importance of delivering emotional and more sophisticated feel. cognitive aspects of colors. Also, the voice O: Applying the same voice to all words delivery with modulation may be used in a phrase that contains words related to color limitedly, only for the key color words. is more enjoyable and comfortable than applying a different voice to only words related to color. B: In the case of music provided in Color Sound, it provides a fairly structured service by varying the pitch and the instruments used, but I think the matching of poetry and color is quite random. Sound will be used for expressing color hues G: The connection between musical and poetry for expressing various color instruments and colors is completely arbitrary dimensions like warm, cool, light, dark, etc. without inevitable reasons. M: There was something I felt while listening to the song, but I don’t think I felt that much about poetry.

5. Discussion and Conclusions Despite the availability of tactile graphics and audio guides, the visually impaired still face challenges in experiencing and understanding visual artworks. In previous works (as described in Section2), musical melodies with different combinations of pitch, timbre, velocity, and tempo were used to distinguish vivid (i.e., saturated), light, and dark colors. However, it was rather difﬁcult to distinguish among warm/cool/light/dark colors with using sound cues only. The way to use poems together when appreciating works has the advantage of enhancing the expressiveness of one work. These motivated us to Electronics 2021, 10, 1064 21 of 24

develop a systematic algorithm to automate the generation of poetry that can be applied consistently to artworks, especially to help visually impaired users to perceive colors in the artworks. Therefore, in this paper, we presented a methodology to create poetry that matched well with a given artwork to ascertain whether a person with a visual impairment can interact with color through sound and poetry together without a complex learning process. Although several researchers have previously performed art–poetry matching, to the best of our knowledge, there is no art–poetry matching for the purpose of conveying different color dimensions such as warmth, coolness, lightness, and darkness. Moreover, there is no previous works that suggest a method of providing poems suitable for the various senses (sight, touch, hearing, smell) of the artwork. In our experiments, an implicit association test was performed to identify the most suitable poem among the candidate poems to represent colors in artwork by finding the common semantic directivity between the given candidate poem with voice modulation and the artwork in terms of light/dark/warm/color dimensions. From the test, we found that the poem P1, for example, exhibited a wider variety of color appearance dimensions, including color temperature, color depth, and color emotion, as well as color lightness that matched well with “Blue sky and night” in Van Gogh’s “The starry night”. Xu et al. [67] propose a memory-based neural model which exploits images to generate poems. Zhang et al. [68] presented a new image-driven poetry recommender system that takes a traveler’s photo as input and recommends classical poems that can enrich the photo with aesthetically pleasing quotes from the poems. They developed a heterogeneous information network and neural embedding techniques. However, they did not take color matters with extensive implicit association tests into account. In this paper, a system usability test was also performed and user experience scores from 15 college student participants were 75.1%, which was comparable with the color–music coding system that received a user experience rating of 74.1%. After training three congenitally blind adults for about one hour, the recognition rate of 18 colors (6 hues and their 3 levels of lightness) using the color–music coding system [17] was 100%. Even though the magic number 5 rule (Nielsen and Landauer, 1993) is vastly known and used for usability testing, the sample size is a long-running debate. Lamontagne et al. [69] investigated how many users are needed in usability testing to identify negative phenomena caused by a combination of the user interface and the usage context. They fo- cused on identifying psychophysiological pain points (i.e., emotionally irritant experienced by the users) during a human–computer interaction. Fifteen subjects were tested in a new user training context and results show that out of the total psychophysiological pain points experienced by fifteen participants, 82% of them were experienced with nine participants. Eye tracking studies take time. For qualitative eye tracking tests where recordings are manually reviewed, 5 users will suffice, but it is necessary to recruit at least 39 participants for meaningful heatmaps and other visualization that aggregate the actions of many users [70]. In the implicit association test done by Greenwald et al. [71], 32 (13 male and 19 female) students from introductory psychology courses at the University of Washington participated in exchange for an optional course credit. Therefore, as a future work, we will further perform scaled experiments on people with visual impairment as a future work, along with experiments to find significant differences in perception of the various levels of the visually impaired for the proposed solution. These studies can enhance the mental imagery experience of color using one or more modalities, such as sounds and poetry, presented in this paper. For practical application, we can identify a set of phrases in poems in a larger sized database (poemhunter.com) that best fit the color dimensions of a given piece of artwork using the same method presented in this paper. In addition, due to the nature of the auditory code, the usability of hearing is higher than the sense of touch and smell, so it can be used in art textbooks for the visually impaired, and it is easy to carry and distribute. When using the mobile phone’s touch screen, colors (hue) can be expressed as sounds, and at the same time, other color dimensions like color temperature and lightness can be expressed by poems, vibrations, or Electronics 2021, 10, 1064 22 of 24

odor, as described in the literature survey [24]. In other words, an integrated multi-sensory platform can convey color images effectively, taking advantage of temperature, vibration, and scent, as well as sound and poetry.

Supplementary Materials: The following are available online at https://www.mdpi.com/article/10 .3390/electronics10091064/s1. Author Contributions: Conceptualization, J.-D.C.; methodology, J.-D.C.; validation, J.-D.C.; formal analysis, J.-D.C.; investigation, J.-D.C.; resources, J.-D.C.; data curation, J.-D.C. and Y.L.; writing— original draft preparation, J.-D.C.; writing—review and editing, J.-D.C.; visualization, J.-D.C.; su- pervision, J.-D.C.; project administration and funding acquisition, J.-D.C. All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by the Science Technology and Humanity Converging Research Program of the National Research Foundation of Korea, grant number 2018M3C1B6061353. Institutional Review Board Statement: The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board of Sungkyunkwan University (protocol code: 2020-11-005-001, 22 February 2021). Informed Consent Statement: Informed consent was obtained from all subjects involved in the study. Acknowledgments: The authors are grateful to Young Jun Sah, who helped in statistical analysis. Conﬂicts of Interest: The authors declare that they have no conﬂicts of interest.

References 1. Calvert, G.A.; Spence, C.; Stein, B.E. The Handbook of Multisensory Processes; MIT Press: London, UK, 2004. 2. Haverkamp, M. Synesthetic Design: Handbook for a Multi-Sensory Approach; Walter de Gruyter: Berlin, Germany, 2012; p. 139. 3. Cottin, M.; Faria, R.; Amado, E. The Black Book of Colors; Groundwood Books Ltd.: Toronto, ON, Canada, 2008. 4. Vinter, A.; Orlandi, O.; Morgan, P. Identiﬁcation of Textured Tactile Pictures in Visually Impaired and Blindfolded Sighted Children. Front. Psychol. 2020, 11, 345. [CrossRef][PubMed] 5. Damásio, A. O Erro De Descartes: Emoção, Razão E O Cérebro Humano; Editora Companhia das Letras: São Paulo, Brazil, 2012. 6. Damasio, A. Feeling & knowing: Making minds conscious. Cogn. Neurosci. 2020, 12, 1–2. 7. Neves, J. Multi-sensory approaches to (audio) describing the visual arts. MonTI. Monogr. Traducción Interpret. 2012, 4, 277–293. [CrossRef] 8. Rector, K.; Salmon, K.; Thornton, D.; Joshi, N.; Morris, M.R. Eyes-free art: Exploring proxemic audio interfaces for blind and low vision art engagement. Proc. Acm Interact. Mob. Wearable Ubiquitous Technol. 2017, 1, 1–21. [CrossRef] 9. Schifferstein, H.N.J. Comparing Mental Imagery across the Sensory Modalities. Imagin. Cogn. Pers. 2009, 28, 371–388. [CrossRef] 10. Literary Devices. Imagery. 2021. Available online: https://literarydevices.net/imagery/ (accessed on 29 March 2021). 11. Trent, A. What Is Synesthesia in Poetry? Available online: https://penandthepad.com/synesthesia-poetry-1824.html (accessed on 29 March 2021). 12. Betts, J. Synesthesia Examples in Literature and Poetry. Available online: https://examples.yourdictionary.com/synesthesia- examples-in-literature-and-poetry.html (accessed on 29 March 2021). 13. Jansson, M. Ekphrasis and digital media. Poet. Today 2018, 39, 299–318. [CrossRef] 14. Wagner, P. Introduction: Ekphrasis, Iconotexts, and Intermediality–the State (s) of the Art (s). In Icons-Texts-Iconotexts; de Gruyter: Berlin, Germany, 2012; pp. 1–40. 15. De Armas, F.A. Visual Culture: Art and Ekphrasis in Early Modern Spain. In A Companion to the Spanish Renaissance; Brill: Leiden, The Netherlands, 2018; pp. 450–470. 16. Hirsch, E. Transforming Vision: Writers on Art; Little Brown and the Art Institute of Chicago: New York, NY, USA, 1994. 17. Yu, W.; Xiaojie, L. On the Expressions in Color in the Poems of William Carlos on the Expressions in Color in the Poems of William Carlos Williams. Foreign Lit. Stud. 2011, 33, 113–120. 18. Koch, K. Rose, Where Did You Get That Red? Random House: New York, NY, USA, 1973. 19. Routman, R. Kids’ Poems: Teaching Third and Fourth Graders to Love Writing Poetry; Scholastic: New York, NY, USA, 2000. 20. Flanders, J. Van Gogh’s Bed. 2015. Available online: https://www.poemhunter.com/poem/van-gogh-s-bed/ (accessed on 4 March 2021). 21. Sexton, A. “The Starry Night” from the Complete Poems of Anne Sexton; Houghton Mifﬂin: Boston, MA, USA, 1981; pp. 53–54. 22. Kennedy, X.J. Nude Descending a Staircase; Doubleday, Incorporated: New York, NY, USA, 1961. 23. Hepner, G. White, Red on Yellow. 2008. Available online: https://www.poemhunter.com/poem/white-red-on-yellow/ (accessed on 3 March 2021). Electronics 2021, 10, 1064 23 of 24

24. Cho, J.D. A Study of Multi-Sensory Experience and Color Recognition in Visual Arts Appreciation of People with Visual Impairment. Electronics 2021, 10, 470. [CrossRef] 25. Cho, J.D.; Quero, L.C.; Bartolomé, J.I.; Lee, D.W.; Oh, U.; Lee, I. Tactile colour pictogram to improve artwork appreciation of people with visual impairments. Color Res. Appl. 2021, 46, 103–116. [CrossRef] 26. Cho, J.D.; Jeong, J.; Kim, J.H.; Lee, H. Sound Coding Color to Improve Artwork Appreciation by People with Visual Im-pairments. Electronics 2020, 9, 1981. [CrossRef] 27. Iranzo Bartolomé, J.; Cho, J.D.; Cavazos Quero, L.; Jo, S.; Cho, G. Thermal Interaction for Improving Tactile Artwork Depth and Color-Depth Appreciation for Visually Impaired People. Electronics 2020, 9, 1939. [CrossRef] 28. Anikin, A.; Johansson, N. Implicit associations between individual properties of color and sound. Atten. Percept. Psychophys. 2019, 81, 764–777. [CrossRef][PubMed] 29. Doppler Effect. Available online: https://science.jrank.org/pages/2147/Doppler-Effect-Doppler-effect-in-sound-waves.html# ixzz6qHpjElX0 (accessed on 29 March 2021). 30. Jonas, C.; Spiller, M.J.; Hibbard, P. Summation of visual attributes in auditory–visual crossmodal correspondences. Psychon. Bull. Rev. 2017, 24, 1104–1112. [CrossRef][PubMed] 31. Cogan, R.D. Sonic Design: The Nature of Sound and Music; Prentice Hall: Upper Saddle River, NJ, USA, 1976. 32. Kandinsky, V. Concerning the Spiritual in Art; Dover Publications: Mineola, NY, USA, 1977. 33. Hamilton-Fletcher, G.; Witzel, C.; Reby, D.; Ward, J. Sound properties associated with equiluminant colors. Multisens. Res. 2017, 30, 337–362. [CrossRef] 34. Giannakis, K. Sound Mosaics: A Graphical User Interface for Sound Synthesis Based on Audio-Visual Associations. Ph.D. Thesis, Middlesex University, London, UK, 2001. 35. Marks, L.E.; Hammeal, R.J.; Bornstein, H.M. Perceiving similarity and comprehending metaphor. Monogr. Soc. Res. Child Dev. 1987, 52, 1–102. [CrossRef][PubMed] 36. Wilms, L.; Oberfeld, D. Color and emotion: Effects of hue, saturation, and brightness. Psychol. Res. 2018, 82, 896–914. [CrossRef] 37. Rowe, A. Audio Dictionary: “Warm” vs. “Neutral” vs. “Bright”. Available online: https://xander51.medium.com/audio- dictionary-warm-vs-neutral-vs-bright-and-the-role-your-brain-plays-in-all-this-2ec761759f7b (accessed on 29 March 2021). 38. Domino, G. Synesthesia and creativity in ﬁne arts students: An empirical look. Creat. Res. J. 1989, 2, 17–29. [CrossRef] 39. Cytowic, R.E. Synesthesia: A Union of the Senses; MIT Press: Cambridge, MA, USA, 2002. 40. Cytowic, R.E. Synesthesia; MIT Press: Cambridge, MA, USA, 2018; ISBN 9780262535090. 41. Maxwell, N.F. “Visualizing Sound: Effects of Pitch Height and Tonality on Luminance Matching. Inquiries J. Student Pulse 2013, 5(10). Available online: http://www.inquiriesjournal.com/a?id=767 (accessed on 28 April 2021). 42. Goller, A.I.; Otten, L.J.; Ward, J. Seeing sounds and hearing colors: An event-related potential study of auditory-visual syn-esthesia. J. Cogn. Neurosci. 2009, 21, 1869–1881. [CrossRef][PubMed] 43. Niccolai, V.; Jennes, J.; Stoerig, P.; Van Leeuwen, T.M. Modality and variability of synesthetic experience. Am. J. Psychol. 2012, 125, 81–94. [CrossRef] 44. Brigg, D. The Dimension of Color. Available online: http://www.huevaluechroma.com/index.php (accessed on 29 March 2021). 45. Artfactory Studio. Two Steps Forward, One Step Back: Colors that Advance and Recede. 2021. Available online: https: //artissima.wordpress.com/tag/temperature/ (accessed on 29 March 2021). 46. Ludwig, V.U.; Simner, J. What colour does that feel? Tactile–visual mapping and the development of cross-modality. Cortex 2013, 49, 1089–1099. [CrossRef][PubMed] 47. Slobodenyuk, N.; Jraissati, Y.; Kanso, A.; Ghanem, L.; Elhajj, I. Cross-modal associations between color and haptics. Atten. Percept. Psychophys. 2015, 77, 1379–1395. [CrossRef][PubMed] 48. Maric, Y.; Jacquot, M. Contribution to understanding odour–colour associations. Food Qual. Prefer. 2013, 27, 191–195. [CrossRef] 49. Kemp, S.E.; Gilbert, A.N. Odor intensity and color lightness are correlated sensory dimensions. Am. J. Psychol. 1997, 110, 35. [CrossRef][PubMed] 50. Fior, A.M. Multisensory integration of visual, tactile, and olfactory aesthetic cues of appearance. Cloth. Text. Res. J. 1993, 11, 45–52. [CrossRef] 51. Kim, Y.J. Can eyes smell? Cross-modal correspondences between color hue-tone and fragrance family. Color Res. Appl. 2013, 38, 139–156. [CrossRef] 52. Hunter, P. Available online: https://www.poemhunter.com/poems/purple/ (accessed on 29 March 2021). 53. Quinn, P.S. Dark Blue Sky. 2007. Available online: https://www.poemhunter.com/poems/blue-sky/page-2/6741601/ (accessed on 3 March 2021). 54. Feldman, S. Blue on Blue. 2013. Available online: https://www.poemhunter.com/poems/Blue/page-2/36448079/ (accessed on 4 March 2021). 55. Vijay, S.R. Yellow Star 2010. Available online: https://www.poemhunter.com/poem/yellow-star/ (accessed on 4 March 2021). 56. Duinker, A. The Wind Has a Blue Tail. 1998. Available online: https://www.poetryinternational.org/pi/poem/4137/auto/0/0/ Arjen-Duinker/The-wind-has-a-blue-tail/en/tile (accessed on 4 March 2021). 57. Aker, K. Blue, Blue and Blue. 2005. Available online: https://www.poemhunter.com/poem/blue-blue-and-blue/ (accessed on 4 March 2021). Electronics 2021, 10, 1064 24 of 24

58. Smith, R.M. Crushed Orange Moon. 2020. Available online: https://www.poemhunter.com/poem/crushed-orange-moon-3/ (accessed on 4 March 2021). 59. Cooper, M. Beneath the Dark Green Grove. 2008. Available online: https://www.poemhunter.com/poems/Green/page-2/1023 6520/ (accessed on 4 March 2021). 60. Safford, A. Color Poems; Brown. 1980. Available online: https://www.csmonitor.com/1980/0310/031059.html (accessed on 4 March 2021). 61. Edwards, B. Green: The Color 2013. Available online: https://www.poemhunter.com/poem/green-the-color-green-things/ (accessed on 4 March 2021). 62. Patel, P. My Imagination of a Purple Sky. 2009. Available online: https://www.poemhunter.com/poem/my-imagination-of-a- purple-sky/ (accessed on 4 March 2021). 63. Sagar, S.A. Green 2007. Available online: https://www.poemhunter.com/poem/green-21/ (accessed on 4 March 2021). 64. Pd. Is Here, Orange (Color Poem). 2006. Available online: https://www.poemhunter.com/poem/orange-colour-poem/ (accessed on 4 March 2021). 65. Song, S. Orange Is Blue. 2006. Available online: https://www.poemhunter.com/poems/Blue-7/page-7/4284787/ (accessed on 4 March 2021). 66. Osgood, C.E.; Suci, G.J.; Tannenbaum, P.H. The Measurement of Meaning; University of Illinois Press: Champaign, IL, USA, 1957. 67. Xu, L.; Jiang, L.; Qin, C.; Wang, Z.; Du, D. How images inspire poems: Generating classical Chinese poetry from images with memory networks. In Proceedings of the 32 AAAI Conference on Artiﬁcial Intelligence, New Orleans, LA, USA, 2–7 February 2018. 68. Zhang, D.Y.; Ni, B.; Zhi, Q.; Plummer, T.; Li, Q.; Zheng, H.; Wang, D. Through the eyes of a poet: Classical poetry recom-mendation with visual input on social media. In 2019 IEEE/ACM International Conference on Advances in Social Networks Anal-ysis and Mining (ASONAM); IEEE: Vancouver, BC, Canada, 2019; pp. 333–340. 69. Lamontagne, C.; Sénécal, S.; Fredette, M.; Chen, S.L.; Pourchon, R.; Gaumont, Y.; Léger, P.M. User Test: How Many Users Are Needed to Find the Psychophysiological Pain Points in a Journey Map. In International Conference on Human Interaction and Emerging Technologies; Springer: Cham, Switzerlands, 2019; pp. 136–142. 70. Dawson, L. Eye Tracking: What Is It for and When to Use It. Available online: https://usabilitygeek.com/what-is-eye-tracking- when-to-use-it/ (accessed on 9 April 2021). 71. Greenwald, A.G.; McGhee, D.E.; Schwartz, J.L. Measuring individual differences in implicit cognition: The implicit associa-tion test. J. Personal. Soc. Psychol. 1998, 74, 1464. [CrossRef]