J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00167 Date: 22-APRIL-14 Stage: I

Gabor Aranyi School of Computing, Teesside University Subliminal Cueing of Selection Middlesbrough, TS1 3BA, UK Behavior in a Virtual Environment

Sid Kouider Laboratoire de Sciences Cognitives et Psycholinguistique (LSCP) E´cole Normale Supe´rieure Paris, 75005, France Abstract

Alan Lindsay The performance of current graphics engines makes it possible to incorporate sublimi- School of Computing, Teesside University Middlesbrough, TS1 3BA, UK nal cues within virtual environments (VEs), providing an additional way of communica- tion, fully integrated with the exploration of a virtual scene. In order to advance the Hielke Prins application of subliminal information in this area, it is necessary to explore in the psy- Laboratoire de Sciences Cognitives et chological literature how techniques previously reported as rendering information sub- Psycholinguistique (LSCP) liminal can be successfully implemented in VEs. Previous literature has also described ´ Ecole Normale Supe´rieure the effects of subliminal cues as quantitatively modest, which raises the issue of their Paris, 75005, France inclusion in practical tasks. We used a 3D rendering engine (Unity3D) to implement a masking paradigm within the context of a realistic scene and a familiar (kitchen) envi- Imtiaj Ahmed Helsinki Institute for Information ronment. We report significant effects of subliminal cueing on the selection of objects Technology (HIIT) in a virtual scene, demonstrating the feasibility of subliminal cueing in VEs. Furthermore, Department of Computer Science we show that multiple iterations of masked objects within a trial, as well as the speed- University of Helsinki ing of selection choices, can substantially reinforce the impact of subliminal cues. This is Helsinki, 00014, Finland consistent with previous findings suggesting that the effect of subliminal stimuli fades rapidly. We conclude by proposing, as part of further work, possible mechanisms for Giulio Jacucci the inclusion of subliminal cueing in intelligent interfaces to maximize their effects. Helsinki Institute for Information Technology (HIIT) Department of Computer Science 1 Introduction University of Helsinki Helsinki, 00014, Finland and Subliminal has recently gained attention in human–computer Helsinki Institute for Information interaction (HCI) as an approach to prime users unconsciously. There is now a Technology (HIIT), Aalto University large consensus on the fact that stimuli remaining below the threshold of con- Helsinki 00076, Finland sciousness (i.e., subliminal stimuli) can have an impact on brain and behavior (Kouider & Dehaene, 2007). Subliminal influences have traditionally been Paolo Negri measured through variants of the masking paradigm (Marcel, 1983). In this Luciano Gamberini paradigm, a stimulus is made invisible by presenting it very briefly (i.e., only a Department of General few tens of milliseconds) and by surrounding it with masking patterns, leading University of Padova Padova, 35131, Italy to chance-level performance on discrimination tasks (objective control of aware- ness) and verbal reports of stimulus absence (subjective control of awareness). David Pizzi Nevertheless, the masked stimulus can influence subsequent decisions by bias- Marc Cavazza* ing selection among alternatives (i.e., subliminal cueing), or by facilitating per- School of Computing, Teesside University formance on subsequent, supraliminal stimuli sharing some relation with the Middlesbrough, TS1 3BA, UK masked cue (i.e., subliminal ). Behavioral findings of subliminal percep- tion revealed that a masked word, digit, or object can have an influence on sen- sorimotor, perceptual, and, under some conditions, even semantic and decision levels, while neuroimaging methods directly visualize the brain activation that it Presence, Vol. 23, No. 1, Winter 2014, 33–50 doi:10.1162/PRES_a_00167 ª 2014 by the Massachusetts Institute of Technology *Correspondence to [email protected].

Aranyi et al. 33 J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00167 Date: 22-APRIL-14 Stage: I

34 PRESENCE: VOLUME 23, NUMBER 1

evokes in several cortical areas (for an extensive review, Ortells, Frings, & Plaza-Ayllon, 2012). The short-lived see Kouider & Dehaene, 2007). nature of the effect of masked subliminal cues imposes While this type of research was traditionally done in additional constraints on the time window in which limited laboratory contexts that are characteristic of ex- cueing will be effective in realistic environments perimental psychology (i.e., using simplified, nonecolog- (Gaillard et al., 2009; Kouider, Berthet, & Faivre, 2011; ical modes of presentation with nothing more than a Brintazzoli, Soetens, Deroost, & Van den Bussche, single stimulus display at a time), the use of virtual envi- 2012). ronments allows for the possibility of studying sublimi- A promising way to address these challenges is to take nal influences in more realistic contexts. Indeed, VEs are known constraints into account when designing the increasingly used to study realistic tasks in a variety of interface for subliminal HCI in virtual environments areas (Fox, Arena, & Bailenson, 2009), supporting the (Pizzi et al., 2012), for example, by embedding sublimi- study of presence and, more recently, research in various nal cues in a goal-directed task while guiding attention areas of neuroscience. Over the years, virtual reality (VR) during its execution. Although there is a wealth of possi- has been considered in research for investigating percep- bilities in terms of applications, research in this direction tion and (Sanchez-Vives & Slater, 2005). seems to lack a systematic approach to study subliminal Although the use of VR to study nonconscious percep- influences. Indeed, many attempts fall short in terms of tion is rare (Hilsenrat & Reiner, 2009), recent advance- the rigorous application of established subliminal per- ments of software and hardware make it possible to syn- ception paradigms from the field of experimental psy- thetically reproduce subliminal cues for a variety of chology, such as controlling carefully for the conscious situations and tasks (Pizzi et al., 2012). visibility of masked stimuli. More effort is needed to Subliminal stimuli can impact users’ behavior in a VE explore subliminal influences in HCI systematically, by and prevent overloading them when a large amount of choosing settings that are representative, clearly specify- data needs to be explored or remembered (Riener, ing the techniques under investigation for specific classes Kempter, Saari, & Revett, 2011; DeVaul, Pentland, & of cues, and determining stimulus visibility. These efforts Corey, 2003). Examples of research include subliminal would ensure that results of experiments could inform cueing in support of online help in a desktop-computer subsequent research with clearer and more generalizable text-editing task application (Wallace, Flanery, & implications, notably in the context of virtual product Knezek, 1991), just-in-time support using experience and electronic shopping (Fox et al., 2009; subliminal cues delivered in a head-mounted display Desmet, Bordenave, & Traynor, 2013; Jiang & Benba- (DeVaul et al., 2003), application in a tutoring system sat, 2004). In particular, it would allow demonstrating (Chalfoun & Frasson, 2011), and aid for visual search the feasibility and potential applicability of including tasks (e.g., McNamara, Bailey, & Grimm, 2008; Bailey, subliminal cues to promote users’ interaction with 3D McNamara, Sudarsanam, & Grimm, 2009). objects in VEs. Affecting behavior in a VE through subliminal stimu- In the present study, we investigated the effects of lation does not come without challenges. Many of these masked indices using a sublimi