Augmented Reality Art and Proprioception: Towards a Theoretical Framework
Total Page:16
File Type:pdf, Size:1020Kb
Augmented Reality Art and Proprioception: Towards a Theoretical Framework Ksenia Fedorova University of California Davis USA/RU [email protected] Abstract allowing them to see alternative interpretations. The poten- Augmented reality (AR) technologies illustrate the paradoxes and tial endlessness of the layers of imaginary figures could be promises of the mechanisms of identification, simulation, and considered an illustration of Jean-François Lyotard's idea imagining that increasingly impact lives and experiences of a of the intrinsic "density" and "thickness" of the visual im- contemporary human. Augmentation implies a layer of infor- age, and so of our sensorium in general. The proposed mation (visual, textual, auditory) that can be called to appear or means of fidelity – 3D images inserted in real time within a disappear on the user's screen and correlate to the objects or situa- certain localized frame – may in the end not be as convinc- tion in actual environment. In what follows, I focus on the theo- ing as they are believed to be (e.g., because of the framing retical framework that informs the aesthetics, working principles itself). Yet this technology furthers the conversation about and critical potential of AR-based artistic projects. I will discuss the ontological (and aesthetic) status of the virtual image, the specific mechanisms AR projects use to explore the effects of its effects and affects, a discussion that has been recently appearing and disappearing, relations of interiority and exteriori- hyper-activated by the advent of Oculus Rift VR., et. al. ty, the visible and the tactile, particularly in their relation to the Duplication of reality through adding another, imaginary feeling of proprioception. layer has a productive effect: however arbitrary and even absurd, the effects of the virtual presence offer a new per- Keywords spective that makes one rethink the status quo of spatial Augmented reality, proprioception, body-schema, body-image, relations and the connotations behind them. An image on sensorium, mixed reality, operability. screen becomes a medium of its own, with its own agency, a projection surface for new meanings and new types of The conceptual strategy of AR art is to display a layer of connections. What is produced is a fresh understanding of the imaginary over the natural visual field; to intermix the space and a sense of co-presence, i.e. the presence of real and the virtual; to make visible and "feelable" the sur- something that leads beyond an individual experience. rounding environment in its spatial depth; and to indicate The examples can include the projects by artists, associ- the potential for modification and intervention. The virtual ated with the Manifest.AR platform, famous for their inter- here is always an addition to the already given: it implies a ventions at the Venice Biennale in 2011, at MoMA, Occu- doubling of realities. AR interfaces are not simply win- py Wall Street, as well as other actions involving political- dows and displays of information to navigate. They are ly charged statements (location-specific virtual replace- operative: manipulating the object on the screen may be ment of iconic monuments, such as the Statue of Liberty, correlated with a mechanical device that maneuvers objects with provocative alternatives). A predominantly formal in the real world. This distinguishes augmented reality exploration of how 3D illusion transgresses the border be- from virtual reality: with augmented reality there is a con- tween immateriality and materiality and becomes opera- tinuity between the physical and the virtual, the "real" does tional and experiential may be exemplified by Fractured not disappear, instead new imaginary objects are projected Visions (2014) by Tamiko Thiel. Reminiscent of avant- onto it. With the experience of AR, the optic serves as a garde formalist experiments, the project attempts to model trigger to instigate specific proprioceptive feelings, e.g. optical experiences of people with a rare disorder (or rather anticipation and desire to experiment with new bodily a symptom) – palinopsia, an effect of multiplication and modes of control. Such an interface challenges the pre- repetition of images in the visual field in a form of super- given conceptions of presence, and opens up to the embod- imposition, 'trailing', 'cascading', etc. The image of the ied exploration of other potential modalities of existence iconic London Shard and buildings around it multiply and and manifestation that have to do with tighter feedback perseverate in the spectator's field of vision. The work aims loop connections between the inside and the outside, feel- to imagine an alternative type of vision, giving "the sense ing and thought. of being 'enclosed' in a spatial, animated visual hallucina- tion." [1] The scientist and clinician collaborating on the The claim behind AR art is often emancipation of the project (Dominic ffytche, KLC Institute of Psychiatry), viewer's ability to discern the constructed nature of reality, noted that AR helps to reveal palinopsia's dynamic nature – the point of gaze, changes in fixation, the timing in relation within the space, i.e. with a potentiality of space. AR inter- to eye movements, the transition in and out of perceptual faces invite the viewers to novel perceptions and interac- consciousness of the palinoptic image. tions within the given location, but also within the space While "Fractured Visions" is about the transformation of where the processes of signification and "making sense" the perspective on existing objects, Will Pappenheimer's happen (what is "expressed" is both on the screen and in Large Reflector (2013) is a virtual object – a sculpture with how the screen is being operated). rotating mirror panels that instead of reflecting the current The "manifestation" of the imaginary figures is triggered surroundings, shows the 360 view in Google Street view of by gesture: the gaze itself becomes a "gestural gaze" the same park location. This mediates the view in a double (P.Lichty). A slight change of the angle of view can make way, locating the viewer between two worlds. the object appear or disappear. The illusionary line of Finally, an example of the conceptual and critical usage connection with it is controlled by the beholder's move- of AR is blemish by CONT3XT.NET, a group of Vienna- ments that activate it by discovering its very existence. For based artists, that was presented as part of the "Invisible instance, in Pappenheimer's work certain sides and "mir- Pavilion" during the Venice Biennale 2011. It addresses ror" images can be accessed only from a particular angle, the very medium of visual production in the digital age and just as a real sculpture should be explored from all the its elementary unit – the pixel (in this case defective, dead), sides through walking around it. Unfamiliarity of the ob- taken within the larger context of the institutional politics ject, or the visual effects over the existing object (as in of image production, representation and circulation in the Fractured Visions) intrigues and stimulates curiosity and world of global contemporary art. blemish focuses on the desire to engage further. As a result, the engagement is not structure of such a potential image and its weaknesses, the so much with the illusory virtual, but with the actual envi- blind spots, emphasizing the unstable nature of reality it- ronment, be it the geometries of a city, its representations, self and the semi-virtual nature of art. The critical dimen- or the world of art (never purely concrete and local). Indi- sion is revealed here through the physical experience itself vidual gesture and movement involved in AR art are a cru- – AR exploration of the renowned site of art consumption. cial part of a dynamic emotional experience of location, These projects may not be the most typical AR art ex- introducing additional, more improvisatory and embodied amples in that, besides Pappenhimer's work, they do not spatio-temporal dimensions that transcend veridical cartog- fully realize the potential of the 'independently' existing raphy. Media content and interaction are resituated into the imaginary objects. Yet, they are critical of both how and real world and thereby into everyday action, incorporating what is being seen, as well as of the figurative approach to techniques of mixed-reality construction. reality in general, its unsteady and yet regulated nature. All of them accentuate the experiential dimension, which The concept of proprioception can be particularly rele- would not be possible without the embodied quality of vant here, as it encompasses the complexity of perceptual perception of AR. relations within both the exterior and the interior of the It can be said that the presence of the virtual object, or body. The term "proprioception" comes from Latin the transformation of vision, like in "Fractured Visions", is propius, or "one's own", "individual", and capio, cepi – "to a pure visual effect. In all these cases, the event in its sen- receive", "to perceive". It implies a sense of one's self felt sory certainty would be impossible without visual percep- as a relational unity of parts of the body most vividly dis- tion. It is viewing that causes the virtual to manifest itself covered in movement and in effort employed in it. With both in its appearance and in potential action. For Lyotard, additional stimulation and training of this sense, what what is important in the work of the visual (or the figural, changes is not only perception of external reality, but a in his terms) is the act of manifestation, of the showing. He sense of one's self – becoming more fluid and porous. A distinguishes the figural from language, i.e. from the dis- sense of proprioception relies on positionality, which sug- cursive regime of signification and meaning construction. gests a subtle equilibrium between a center (our individual The figure gives language "density" and "thickness," as bodies) and the periphery (our immediate environments).