A Situated Cognition Perspective on Presence
Antonella Carassa ([email protected]) Faculty of Communication Sciences, University of Lugano via G. Buffi 13, 6900 Lugano, Switzerland
Francesca Morganti ([email protected]) Applied Technology for Neuro-Psychology Lab, Istituto Auxologico Italiano via Pelizza da Volpedo 41, 20149 Milano, Italy
Maurizio Tirassa ([email protected]) Department of Psychology and Center for Cognitive Science, University of Torino via Po 14, 10123 Torino, Italy
Abstract sense of being placed somewhere different from the actual physical location. Sheridan, in particular, has defined virtual During interaction with computer-based 3-D simulations like presence as the subjective feeling or mental state in which a virtual reality, users may experience a sense of involvement subject has the belief of being "physically present with called presence. Presence is commonly defined as the subjective feeling of "being there". We discuss the state of the visual, auditory, or force displays generated by a computer". art in this innovative research area and introduce a situated Heeter (1992) has defined an environmental presence which cognition perspective on presence. We argue that presence is yielded by the perception that an environment exists that depends on the proper integration of aspects relevant to an seems to consider you as present and modifies depending on agent's movement and perception, to her actions, and to her what you do. Witmer and Singer (1998) also have taken conception of the overall situation in which she finds herself, presence to be due to immersion, but have linked it to the as well as on how these aspects mesh with the possibilities for tendency to direct attention toward selected information that action afforded in the interaction with the artifact. We also aim at showing that studies of presence offer a test-bed for be meaningful to the individual. Presence would then be different theories of situated cognition. similar to selective attention, and the sense of presence would be yielded by the allocation of attentional resources. Keywords: Situated cognition; presence; virtual reality; According to these authors, both involvement and affordance; action; artifacts immersion are needed to experience presence. Their approach, while focusing on immersive properties, also Introduction emphasizes the role that attention-directing role of activity within complex interactive situations. The development and the diffusion of computer-based The specific role that interaction with technology plays in interactive simulations, like 3-D videogames or virtual creating presence has been firstly considered by Lombard reality, has been accompanied by an increasing perception and Ditton (1997), who have defined presence as the of the importance of the sense of presence and involvement "perceptual illusion of non-mediation". In particular, that users may experience while interacting with them. according to Lombard (2000), presence should be divided Presence is commonly defined as the subjective feeling of into those aspects which involve the perception of a physical "being there" (Riva, Davide & IJsselsteijn, 2003; Slater, environment (where the sensory features correspond to Usoh & Steed, 1995; Witmer & Singer, 1998). Several those of the physical world), those which involve the authors have considered this feeling as mainly deriving from perception of social interaction (where the social features the immersion in a virtual environment (Schubert, correspond to those of the physical world), and those which Friedmann & Regenbrecht, 1999a; Schubert, Friedmann & involve both. Regenbrecht, 1999b; Slater & Wilbur, 1997). In their view, In this perspective, presence is taken to occur when a presence results from subjective involvement in this kind of person misperceives an experience mediated by technology highly interactive virtual environment. Presence would be as if it were a direct (that is, non-mediated) one. Presence, strong inasmuch as the virtual system enables an inclusive, then, would not be intrinsic to technology, but would vary extensive, surrounding and vivid illusion: the immersive depending on how much the user acknowledges the role of quality of a virtual reality system would be enhanced by the technology. It could therefore be yielded by different kind perceptive features and the proprioceptive feedback of technologies. provided by the system. The importance of activity in the support and the Within this perspective, different authors have developed enhancement of presence in virtual reality has been different conceptions of presence. Sheridan (1992) and investigated by Flach and Holden (1998), who have Zeltzer (1992), for example, have described presence as the emphasized the necessity that interaction with objects be
384
introduced in virtual environments. On a similar vein, between proto presence (that is, the distinction between self Zahorik and Jenison (1998) have focused on the role of and not-self), core presence (that is, the distinction between plausibility in perception/action behaviors; the latter are the self and the present external world), and extended dealt with in terms of affordances. presence (that is, the distinction of self with respect to the Thus, while studies in the area began within a purely present external world). These authors stress that the sense technological, artifact-centered perspective, the need for a of presence is a direct function of these three layers (the more psychological, human-centered framework has been more they are integrated, the more we feel present) and that, felt with increasing strength. in the experience of optimal presence, biologically and In an attempt to combine immersion-based theories with culturally determined cognitive processes work in harmony activity-based ones, Sheridan (1999) has built his to focus all the levels of the self on a significant situation in Estimation Theory on the assumption that we can never the external world, be it real or virtual. have true knowledge of objective reality; instead, we are Finally, Biocca (2001) has claimed that the issue of continuously making and refining a mental model which presence may be most fruitfully approached via philosophy estimates reality. This process is made possible by our of mind. Specifically, he has suggested that the issue of sensing reality and interacting with it. Immersion in virtual presence opens the door to related problems in the science reality is a source of stimuli, starting from which users of human consciousness, notably the mind-body problem. would create a mental model of the virtual environment and The problem of presence bridges the philosophy of mind of how they relate to it. It would be the structure of this and the philosophy of technology on the issue of mediated mental model that determines whether or not users embodiment, that is, on the fuzzy boundaries between the experience a sense of presence. Thus, even when the users body and its technological extensions. are uncertain about the reality of their perceptions in the We agree that a proper treatment of the issue of presence virtual environment, such perceptions would anyway be in virtual reality should begin with the acknowledgment that close relatives of those they have in the "real" physical presence is a major feature of human consciousness, and the world. hub of any interaction with natural as well as artificial Mantovani and Riva (1999) have been the first to environments. emphasize the importance of the freedom that users may The environments in which human beings live are have in their interaction with the virtual environment, as endowed with a huge component of artificiality: virtual well as the need that the social and cultural dimensions of reality, as it has been developed in the last two decades, is a action in both the physical and the simulated world be recent and particularly sophisticated expression of our basic thoroughly considered. According to them, every action — biological disposition to create artifacts as supports for and especially those performed in a simulated world — intrapersonal and interpersonal activities. takes place within a framework of meanings that are Human beings interact with and through artifacts — and, grounded in the user's cultural background. It is based on the in more general terms, with their world or worlds — in information found in this cultural framework that each ways that depend on the representations that they entertain specific user develops her sense of presence in interaction. about them. We take representations to be conscious mental Culture shapes the individual minds and organizes their states (Searle 1992). Representations are essentially about ways of knowing and acting in their social and physical spaces of possible actions (affordances) and are thus colored environment (Mantovani & Riva, 2001). When the with different degrees of desirability and emotion (Tirassa, information provided by the environment is ambiguous to in press; Tirassa, Carassa & Geminiani 2000). They the individual, as may happen with virtual reality, the originate in the interaction between an individual, who is ambiguity can be resolved or reduced with the aid of endowed with her personal history as well as with the culturally based information that helps the user to reinterpret biological history of our species, and a perceived, desired, and understand the overall interaction. remembered or imagined environment. Riva, Waterworth and Waterworth (2004) have pointed Our basic tenet is that the mind should not be conceived out that a culture-based view of interaction supports a of as an ideal, abstractly defined cognitive system (possibly perspective on presence as " the referencing of our a computational or information-processing one) which is perception to an external space beyond the limits of the "implemented" in a separate and interchangeable material sensory organs themselves". According to this perspective, structure that physically realizes its activities. We think it is the capacity of distal attribution — that is, the capacity instead that the mind is a biological property of the body, that humans have of distinguishing between their own body with which it is in a relation of coevolution and and the external world — that allows for the sense of codetermination. presence in simulated as well as in non simulated worlds. Even if the experience of presence is a unitary feeling, it can Affordances as Representations be conceptually divided in three different layers, The human mind/body has evolved for generating an phylogenetically different and closely related to the three adaptive interaction with the world, and in particular for levels of self discussed by Damasio (1999). Riva, planning actions which be able to modify the environment Waterworth and Waterworth (2004) thus distinguish in order to fulfill individual aims and desires. It is thus
385
reasonable to accept the suggestion, originally advanced primarily based on perceptual-motor integration, a deeper from within ecological psychology (Gibson, 1979), that an analysis reveals that a much wider array of cognitive individual perceives the world not in terms of objective, capacities is involved. abstract features, but directly in terms of spaces of possible Think of a person who is going to prepare a cake, and actions, that is, of affordances. An affordance is not defined imagine that he choose to use a whisk to prepare part of the univocally based on the agent's features or of the world's dough. features alone. It is instead a relational notion: possibilities The whisk, a small object made of curved pieces of wire for action depend on the encounter of the characteristics of with a handle, affords a variety of movements that depend the two poles of the interaction, and are shaped by the on how our physical (manual in this case) capacities mesh overarching activity in which the agent is involved (Carassa, with features of the object like its shape, dimensions, Morganti & Tirassa 2004). structure, and rigidity. The point is that the very object In our reading of Gibson's work, what the world puts in immediately affords one particular movement to someone the agent/environment interaction is the resources and the who possess a specific expertise. This peculiar affordance constraints to which the animal's control systems must does not depend on perceptual-motor abilities alone: it conform, such as the invariants of the optical flow (which is requires the capacity to recognize what the specific function also part of the reason why Gibson talks of "direct of the artifact is within the practices aimed at preparing perception"). What the animal puts in the interaction is its certain specific recipes. An experts is "invited" by the whisk own nature, which makes a certain configuration of to perform a quite technical movement: to handle it in an surfaces, a certain texture of the layout, and so on, take the oblique position so to be able to execute a quick light mental shape of a certain affordance. sweeping movement. If the animal's nature includes a mind like ours, the The management of such movement depends on the traditional concept of affordance has to be reconsidered in creation, the maintenance, and the moment-by-moment the light of what we actually know about human cognition. reactivation of sensorimotor schemes that "tell" the agent The dimension of interaction which have been more how to appropriately program his movements in the specific thoroughly discussed in recent research is perceptual-motor situation in which he finds himself, what sorts of feedback coordination. In investigating how affordances develop to expect from the world, and so on. That way, his cognitively and behaviorally, particularly in the first years mind/body will "know" what muscular power to exert in of life, the focus is thus on activities like crawling or order to achieve that particular movement; and, if he reaching for and grasping objects; and the problems exercises such power correctly, his mind/body will "take it typically addressed concern the locomotory control, based for granted" that the world will respond in agreement with on information contained in the optical flux, or the adaptive certain invariant rules of the physical world. Actually, it is control of action through the integration of proprio- and also on the grounds of such feedback that he will be able to estero-ceptive information. know how he is executing his movement, which point in a Research has been less concerned with the cognitive particular sequence of movements he has reached, and so dimensions of interaction. However, these dimensions are on. identified as the crucial ones for treating the issue of The whisk, like any artifact, is an embodied project whose presence in the literature on virtual reality. It has been functions only exist in the minds of the designer and of the claimed, for example, that "it is necessary to add layers of user (Tomasello, 1999; Mantovani, 1996; Norman, 1988). meaning to perceptual objects" (Biocca, 2001), and that it is Let us consider again how a potential user may be able to necessary to study "the interplay of bodily and cognitive grasp what function the whisk incorporates. He will be able processes" (Schubert, Friedman & Regenbrecht, 1999b) and to infer the designer's intentions in the light of a rich the mental models that people entertain when interacting background of knowledge. For example, if he knows that with virtual environments (Biocca, 1997; IJsselstein, 2002; certain types of cakes contain a soft and creamy component, Riva, Waterworth & Waterworth, 2004). he will be able to imagine the possible make-up of a specific In this vein, we will discuss the cognitive nature of tool designed to obtain this effect, as well as to recognize an affordances as representations situated in an unfolding object which has been designed for that purpose. interaction. We will use as an example the case of an agent Circularly, the whisk designer will ground her work on engaged in cooking, an activity that necessarily requires the the knowledge that she thinks she can safely attribute to a use of artifacts. (anonymous, but decently expert) user, and will probably try Cooking is pervasive in everyday life; it is grounded in a to make the affordances that the tool incorporates as evident history-long cultural tradition and embued with values and as possible. social meanings. In our culture, the art of cooking is Thus, an affordance does not result from a perceptual act particularly appreciated and can be performed with a rich alone, nor from a simulation of perceptual-motor abilities repertoire of practices and specialized tools. Each tool has alone. It is instead a representation in which different been designed for a specific function, tailored to a class of aspects are meshed: cultural aspects (culinary culture), activities like chopping, cutting, smashing and so on. While, representation of function (a tool-to-whip-a-cream), and at first glance, the use of such artifacts may be viewed as somatic/functional aspects (the human hand's articular
386
mobility and grasp). Moreover, for an individual engaged in overall situation in which she finds herself, as well as on an unfolding interaction, the landscape of affordances is how these aspects mesh with the possibilities for action shaped by his motives, interests, values and long-term goals afforded in the interaction with the virtual environment. and desires. All these elements give meaning to what is We have distinguished three levels in the interaction of an perceived and done moment by moment. agent with her world, be it real or virtual: that of the Meaning does not arise only from the sensibility to situation, that of the action, and that of body movement and appreciate what is currently happening and from the ability perception (Carassa, Morganti & Tirassa 2004). These to react in a fluid and savvy way: it depends on our levels are not reducible to one another; instead, each of them conceptualizations of ourselves as immersed in a global "contains" and gives meaning to the subsequent one, which situation, which is in turn embedded in our personal history. in turn realizes it, in a circular relationship of In this sense, our feeling of presence is not determined by codetermination. the physical place to which we are immediately tied by Normally, an agent will not think of her movement in perception, be it natural, artificial, or virtual; instead, it is terms of a motor sequence (unless, of course, she has any grounded in a meaningful situation that stretches in the past reason to do so, like when learning a physical exercise, in and faces future. The meaning of the situation, which is which case the motor sequence may be the very action or conceptual in its nature (Clancey, 1997), creates a bundle of situation). Instead, she will choose and perform actions affordances that become relevant within a specific whose goals are part of a broader situation, which she interaction. represents as the activity, or the weave of activities, in In an individual's subjective perspective, each action that which she is participating at each moment. he performs plays a role within a narrative that he tells These activities are, in their turn, supported by goals, himself concerning what is happening, what he is doing values, knowledge, and roles that give them meaning, there and why, with what future perspectives, and so on. boundaries, histories, and possible directions of More precisely, each action that he performs plays a role development. within a complex weave of such narratives, that end up Therefore, an individual will represent herself not as a giving a meaning to his being there, in that place, in that monad with no history who "behaves" in an objectively very moment, choosing to perform a certain action, as well given world, but as an agent who carries on a narrative as to the specific body movement which ultimately shapes about herself in the world. What is of interest to her is to the material counterpart of his mental state. Each such follow complex flows of meaning relevant to the different narrative may be viewed as a choreography (Clancey, 1997) choreographies in which she finds herself. Her in which the agent features as the protagonist; each has an representations and actions create her participation in such intrinsically autobiographical and social nature, and the choreographies from moment to moment. overall weave thus results from the whole previous history How does this conception of mind and agency, a of that individual (which includes, of course, his current and constructivist and interaction-based one (Tirassa, in press) past hopes and plans for the possible futures). affect our conception of experience and presence in virtual reality? Situated Presence When the interaction is such that a good feeling of presence is generated and maintained, several other things Our view is different from the mainstream view of will become possible. Firstly, the mind may compensate "embodied cognition" that has led Schubert, Friedmann and with its own capacities, at least to a certain extent, for the Regenbrecht (1999b) to talk of "embodied presence" in the "low fidelity" of the simulated world. As we have said following terms: "Presence develops from the representation above, what makes the difference is not the technological of navigation (movement) of the own body (or body parts) perfection of the virtual environment, but the type of as a possible action in the virtual world. Presence is the interaction that it affords. outcome of media perception. In the process of developing Secondly, just as action support presence, so does presence a mental model of the virtual three-dimensional presence support action. The feeling of presence is space is constructed, consisting of the possible actions in satisfactory when the user manages to make an overall sense this space. The possible actions of the body are central in of her interaction with the environment. When this happens, this model. Stimuli from the real environment must be she will also be able to make such interaction useful and suppressed for presence to emerge. The more the mediated interesting for her future narratives: in simple terms, she stimuli follow embodied constraints (e.g., coupling with will be able to learn something. body movement), the easier is the construction. Because the Thus, in experiencing a virtual reality environment, the environment is perceived in terms of embodied action user will bring with herself everything that she has been up (affordances in our terms), a feeling equivalent to the feel of to that moment. By the same token, her experience with the memory mentioned above develops. This is what we call the media will add to her "cognitive history". This may mean sense of presence." that she will have acquired knowledge (e.g., concerning the In our view, instead, presence depends on the proper Qumran scrolls, or how to fly an airplane), or that she will integration of aspects relevant to an agent's movement and have spent a few hours shooting nasty green aliens that want perception, to her actions, and to her conception of the
387
to invade the Earth, or, in the worst case, that she will have Acknowledgments suffered from cybersickness — even this is an experience, Antonella Carassa and Francesca Morganti were funded by however unpleasing, that will affect her possible futures. the Italian Ministry of Education, University and Research What the designer does is to create an envelope within (MIUR), Cofin 2003 prot. 200311 9035 ("Fattori percettivi e which the interaction with the virtual environment may cognitivi in compiti di navigazione spaziale in ambienti di belong to a weave of narrative meanings. The goal of such realtà virtuale"). Francesca Morganti was also funded by the enterprise is not intrinsic to the virtual environment, but is European Union, project on I-Learning (FP5 IST/FET 2001 born out of the structural coupling between the user and the - 38861). Maurizio Tirassa was funded by the University of environment — or, as happens, between the user, the Torino, Fondi individuali di ricerca (Quota ex-60%) for the environment, and a supervisor or a tutor who guides the year 2004. interaction. The kernel of our position is that the environment, even a References virtual one, has a subjective, rather than objective, nature. Biocca, F. (1997). The cyborg's dilemma: Progressive The classic dichotomy between an external world, which is embodiment in virtual environments. Journal of objectively given, and an internal world, which mirrors it Computer-Mediated Communication, 3, 2.
388
Mantovani, G., & Riva, G. (2001). Building a bridge Slater, M., Usoh, M, & Steed, A. (1995). Taking steps: The between different scientific communities: On Sheridan's influence of a walking technique on presence in virtual. eclectic ontology of presence. Presence, 10, 5, 537-543. ACM Transactions on Computer-Human Interaction, 2, 3, Norman, D.A. (1988). The psychology of everyday things. 201- 219. New York: Basic Books. Slater, M, & Wilbur, S. (1997). A framework for immersive Riva, G., Davide, F., & IJsselsteijn, W.A. (2003). Being virtual environments (FIVE): Speculations on the role of there: Concepts, effects and measurements of user presence in virtual environments. Presence, 6, 6, 603-616. presence in synthetic environments. IOS Press. Tirassa, M. (in press) Agencies. In L.A. Pérez Miranda & Riva, G., Waterworth, J.A., & Waterworth, E.L. (2004). The J.M. Larrazabal (Eds.), [Title to be announced]. layers of presence: a bio-cultural approach to Dordrecht: Kluwer. understanding presence in natural and mediated Tirassa, M., Carassa, A., & Geminiani, G. (2000). A environments. Cyberpsychology and Behavior, 7, 402- theoretical framework for the study of spatial cognition. 416. In S. Ó Nualláin (Ed.), Spatial cognition. Foundations Schubert, T.W., Friedmann, F., & Regenbrecht, H.T. and applications (pp. 19-31). Amsterdam & Philadelphia: (1999a). Decomposing the sense of presence: Factor Benjamins. analytic insights. Presented at the 2nd International Tomasello, M. (1999). The cultural origins of human Workshop on Presence. University of Essex, UK, April 6- cognition. Harvard, MA: Harvard University Press. 7. Varela (1990) Il corpo come macchina ontologica. In M. Schubert, T.W., Friedmann, F., & Regenbrecht, H.T. Ceruti & F. Preta (Eds.), Che cos'è la conoscenza. Bari: (1999b). Embodied presence in virtual environments. In Laterza. R. Paton & I. Neilson (Eds.), Visual representations and Witmer, B.G., & Singer, M.J (1998). Measuring presence in interpretations (pp. 268-278). Berlin: Springer. virtual environments: A presence questionnaire, Presence, Searle, J.R. (1992). The rediscovery of the mind. 7, 3, 225-240. Cambridge, MA: MIT Press. Zahorik, P., & Jenison, R.L. (1998). Presence as being-in- Sheridan, T.B. (1992). Musings on telepresence and virtual the-world. Presence, 7, 1, 78-89. presence. Presence, 1, 1, 120-126. Zeltzer, D. (1992). Autonomy, interaction, and presence. Sheridan, T.B. (1999) Descartes, Heidegger, Gibson, and Presence, 1, 1, 127-132. God: Towards an eclectic ontology of presence. Presence, 8, 5, 551-559.
389