Fooling the Brain with the Augmented Mirror
Total Page:16
File Type:pdf, Size:1020Kb
Holger T. Regenbrecht* Beyond the Looking Glass: Fooling Elizabeth A. Franz Graham McGregor the Brain with the Augmented Brian G. Dixon Mirror Box Simon Hoermann University of Otago Dunedin, New Zealand Abstract Video mediated and augmented reality technologies can challenge our sense of what we perceive and believe to be real. Applied appropriately, the technology presents new opportunities for understanding and treating a range of human functional impair- ments as well as studying the underling psychological bases of these phenomena. This paper describes our augmented mirror box (AMB) technology which builds on the potential of optical mirror boxes by adding functions that can be applied in therapeutic and scientific settings. Here we test hypotheses about limb presence and perception, belief, and pain using laboratory studies to demonstrate proof of concept. The results of these studies provide evidence that the AMB can be used to manipulate beliefs and perceptions and alter the reported experience of pain. We conclude that the system has potential for use in experimental and in clinical settings. 1 Introduction The line between what is real and what can be computer generated is becoming increasingly blurred with modern technology (cf. IJsselsteijn, de Kort, & Haans, 2005). Augmented reality allows us to create a mirror image of what is real (let us say human hands), and present this image in a virtual envi- ronment. Augmentation adds to the blurring effect, since it can raise doubt about whether we think we are looking at our hands or at some enhanced or manipulated version of them projected on a screen. This effect is related to the phenomenon of neuroplasticity, the brain’s ability to adapt its functions and activities in response to environmental and psychological factors (Doidge, 2010). In our view, neuroplasticity is mediated by beliefs, perceptions, sensa- tions, and emotions. It is the brain’s ability to act and react in ever-changing ways, through thought and activity based on environmental input, which has led to the novel system that we describe in this paper. Our system is an augmented development of the standard optical mirror box (Ramachandran, Rogers-Ramachandran, & Cobb, 1995; Moseley, Gallace, & Spence, 2008) and has enabled us to explore the mirroring process and its potential application in experimental and clinical settings. The issues that led us to develop an augmented mirror box are presented by way of background. We then provide a detailed description of our new system approach and the capabil- ities that we are currently exploring. Finally, for proof of concept, data are Presence, Vol. 20, No. 6, December 2011, 559–576 ª 2012 by the Massachusetts Institute of Technology *Correspondence to [email protected]. Regenbrecht et al. 559 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00082 by guest on 27 September 2021 560 PRESENCE: VOLUME 20, NUMBER 6 presented from two studies involving control partici- ual appearance of two limbs (bimanual coupling) in pants and a case study on a patient with severe (complex action. It was hypothesized that this form of virtual regional) pain.1 movement could be used to engage the brain’s proc- esses associated with the other (impaired) limb, thereby reducing spatial and motor impairments. Positive results were achieved through various case studies that 2 Background followed showing a reduction of pain in amputees (Ramachandran & Rogers-Ramachandran, 1996), peo- Our augmented mirror box (AMB) system derives ple with chronic pain (Tichelaar, Geertzen, Keizer, & from sensory and motor research with human beings van Wilgen, 2007; Rudd, Schreuders, & Tam, 2008), that involves the use of therapeutic devices to ameliorate and those with wrist fractures (Altschuler & Jeong, unilateral (one sided) sensory and motor impairments. 2008). Enhanced motor output was also achieved in For example, research on the effects of limb amputation patients with unilateral stroke (Gaggioli et al., 2005; has revealed that some amputees experience phantom Jang et al., 2005; Dohle et al., 2008). Franz’s lab was pain in the stump of the missing limb. Other amputees able to demonstrate an initial proof of their concept report a phenomenon referred to as phantom limb using a larger group of control participants who experi- movement. This phenomenon does not involve pain as enced enhanced bimanual coupling using the optical such, but rather a vivid experience that the missing limb mirror box (Franz & Packman, 2004). Prototypes of can still move, even as a result of the participant’s own mirroring using virtual reality, along the lines of our volition (Carlen, Wall, Nadvorna, & Steinbach, 1978). system, are a more recent development (Giraux & This effect is similar to the rubber arm/hand illusion, Sirigu, 2003; Sveistrup, 2004). where individuals can develop ownership of an artificial We developed a therapy treatment apparatus and (rubber) limb under certain circumstances and which methodology that can represent the mirrored realism of can also be achieved with virtual limbs (Yuan & Steed, one’s own limb with the flexibility of control that a vir- 2010). tual reality approach offers. As we will show, such a vir- In the perceptual-motor domain, Franz and col- tual reality system actually allows for particular manipula- leagues have demonstrated a form of spatial coupling tions and treatments that a standard OMB cannot that typically occurs between the limbs of the body, provide. Our system combines customizable and special- resulting in a tendency for the limbs to move in similar ized off-the-shelf (COTS) hardware and software. The (often mirror symmetrical) patterns (Franz, 1997). system is designed to provide (a) a well-controlled envi- Combining these findings, Franz and Ramachandran ronment to test our hypotheses about perceptions and (1998) tested amputees on a bimanual task to examine beliefs; and (b) a reliable and robust apparatus for use in whether bimanual coupling occurs even if a limb is therapeutic settings. missing, by measuring the movement output of a In what follows, we explore how an OMB works and healthy limb during different conditions of imagined its uses in therapeutic and scientific settings, how the phantom limb movement. Bimanual coupling still shortcomings of an OMB can be mitigated, and how occurred, suggesting that some forms of coupling new visual effects can be achieved through augmenta- depend on central, rather than peripheral or physical tion. Proof of concept studies comparing the AMB with processes. A technique using an optical mirror box the standard OMB are presented, including how we (OMB) was developed during the course of that work. tested our assumptions about possible changes in per- The mirror box enabled the researchers to project an ceptions and beliefs when using our system and applica- image of the healthy limb moving, thus giving the vis- tions that extend beyond optical mirroring. Finally, we 1. An earlier version of this paper was made available as a technical consider how the AMB could be developed for use in report (Regenbrecht, Franz, McGregor, Dixon, & Hoermann, 2011). experimental and clinical settings. Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_a_00082 by guest on 27 September 2021 Regenbrecht et al. 561 3 The Optical Mirror Box The optical mirror box is well known as a therapeu- tic device that assists in the treatment of unilateral sen- sory and motor impairment, such as phantom limb pain and the early and intermediate stages of complex re- gional pain syndrome (CRPS; Giummarraa, Gibson, Georgiou-Karistianisa, & Bradshaw, 2007). We theorize on the basis of evidence that the optical mirror box uses an ordinary mirror to confuse our senses about the own- ership of our hands, or our limbs in general. For exam- ple, in phantom-limb pain management, the stump limb of an amputee is placed behind a mirror while the healthy limb is reflected in the mirror. If the client moves Figure 1. Bimanual movement in an optical mirror box setup. both limbs (real and illusory movement) in a simultane- ous way, the reflected healthy limb is often taken for the other one. This bimanual coupling effect is reported in for experimental manipulation and the realism of the studies with normal controls, where the participants phantom representation. In many cases, the phenom- actually produce enhanced degrees of coupling of simul- enologically experienced phantom limb differs sub- taneous movements with the use of a mirror box com- stantially from the limb before amputation, and from pared to without (Franz & Packman, 2004). the remaining ‘‘intact’’ limb. The lack of resemblance However, in these studies, participants are aware that between the phantom limb and reflected image may it is their own limb being reflected, given that they can diminish or inhibit the therapeutic value of the inter- always see the reflection. For some patients in mirror vention. therapies that have been developed to date, this aware- ness does not pose problems. However, it leaves open Our research seeks to extend the therapeutic possibil- the question of whether the effects of the mirror box are ities of the OMB by allowing the illusion to be mediated a result of sensory influences, or possible higher order in different ways, using visual augmentations and altera- cognitive processing such as belief or expectation. Previ- tions, for example. The technology involved for this pur- ous work on neurologically normal controls without uni- pose is described in the following section. lateral impairment suggests that the mirror box fools the brain into thinking it sees two hands moving, resulting 4 The Augmented Mirror Box System in a fusion of the visual onto the proprioceptive inputs (Franz & Packman, 2004). With the optical mirror box This section reports on the development of a com- it is clear that this so-called illusion of bimanual move- puter-assisted AMB setup to test hypotheses about limb ment exists only in the mirror (see Figure 1).