’s note The Art Installation Endocytosis Evolving from a Flat Land into a Three-Dimensional World

Á g u e d a S i m ó

Virtual reality systems are an ideal platform for exploring spatial [8]. In addition, stereopsis is a strong emotional and aesthetic effects because of their ability to combine stereo imaging techniques factor in the [9] and an essential contributor to the and interactive real-time graphics. They allow the creation of artworks sense of immersion in VR [10]. that, on the one hand, exhibit a dynamic organization of the ABSTRACT environment’s spatial depth and, on the other, create an interaction Background with the stereoscopic optic flow. In this article, the author discusses the advantages of horizontal stereoscopic displays and describes The use of stereo imaging techniques in computer graphics how she has used the Responsive Workbench to display the evolution offers exciting possibilities to interested in the percep- of a flat land into a three-dimensional world in her artwork Endocytosis. tion of depth, as these techniques allow the exploration of She uses endocytosis—a fundamental cellular trafficking process that moves material into the intracellular space—as a metaphor for this spatial effects that would otherwise be impossible to achieve evolutionary process. [11]. In this respect, VR systems offer the advantage of en- hancing depth discrimination with the use of head-tracking devices [12]. The use of VR systems with real-time computer Over the centuries, artists have represented the world on flat graphics, stereoscopic visual displays and tracking devices surfaces using pictorial cues to create the illusion of depth. can be traced back in art to the beginning of the 1990s [13]. However, it was not until the nineteenth century—when However, the high costs and technical difficulties of produc- Charles Wheatstone described the basis of stereopsis [1]— ing and exhibiting these artworks have limited the explora- that images could be produced that appeared to be three- tion and application of these VR in art. Most dimensional (3D). Wheatstone invented the stereoscope VR artworks use a vertical stereoscopic projection system, and demonstrated the link between binocular rivalry and very much resembling the we hang on walls, or stereoscopic depth perception [2]. The coupling of stereos- even a television set. On the other hand, some notable art copy with photography resulted in a new way to visually de- and design projects that focus on modeling, life or pict spatial depth [3,4]. Ever since then, stereoscopic images fine-grained manipulation of objects have used horizontal have excited artists and scientists, for “there is a quality of stereoscopic projections [14,15] such as by means of the Re- magic in the experience of depth from stereopsis. . . . a vivid sponsive Workbench (RW) developed by Wolfgang Krueger sense of the space between and taken up by objects in the and Bernd Fröhlich [16]. Prior to the development of the world” [5]. RW, computer-generated stereoscopic images were usually During the twentieth century, stereo imaging techniques displayed on head-mounted displays or projected onto one improved with the emergence of computer graphics and or various vertical screens, with some special cases using a viewing devices that evolved from the basic stereoscopes of combination of horizontal and vertical screens (the CAVE) David Brewster [6] to the advanced 3D displays used in vir- [17]. Other VR systems use tilted screens (the ImmersaDesk) tual reality (VR). or L-shape projections with two orthogonal screens (the Ho- Stereopsis is doubtlessly effective and intuitive for under- lobench) [18]. standing space [7], therefore it is a key factor in data visual- Vertical VR systems provide a natural interaction that fa- ization and thus of increasing interest in computer graphics cilitates navigation (e.g. driving) through the environment toward the horizon. Horizontal VR systems emphasize

Águeda Simó (artist, teacher), University of Beira Interior, Department of fine-grained interaction with the objects displayed on the Communication and Arts, Covilhã, Portugal. Email: . horizontal plane (above or below the tabletop) rather than See for supplemental files associated with navigation through the environment [19]. In this respect, this issue. the tabletop framework provides a physical affordance for

124 LEONARDO, Vol. 51, No. 2, pp. 124–127, 2018 doi:10.1162/LEON_a_01265 ©2018 ISAST

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_01265 by guest on 28 September 2021 in his seminal book Flatland [22]— and to create a continuous flow between the virtual and the physi- cal spaces that merge dynamically and seamlessly. This coalescence between the virtual and physical spaces takes place by means of the rigid visual interface of the RW that becomes a virtual membrane to the third dimension. The passage from a two-dimensional world to a three-dimensional world implies, on the one hand, our ability to re- dimension the space; on the other hand, it implies our limitations in perceiving higher, or lower, dimen- sions and accepting reality as a con- struction of our mind. The tabletop acts like a mem- brane interface: The side facing the user is the interior of the cell, with a negative parallax (popping out from the screen), while the other side (below the tabletop) constitutes the cell’s surroundings or outside world, having a posi- tive parallax (receding behind the screen). The exchange of material between both sides occurs through the traffic of cargo inside spheri- cal vesicles. I digitally painted the tabletop black, except for the circles upon it, which are displayed with Fig. 1. Users interacting with Endocytosis. (© Águeda Simó) zero parallax (Color Plate B). These circles represent specific membrane moving objects around the table as well as lifting and push- domains that grow and become spheres. They protrude from ing them down (Fig. 1); moreover, users can move around the RW’s flat surface to enter the three-dimensional world the table and acquire different viewpoints of the [20]. above it (the cell) and, in doing so, they open peepholes on Horizontal VR systems are particularly good at providing a the tabletop, that is, into the universe surrounding the cell. bird’s-eye view together with binocular parallax on the hori- Users can comfortably lean on the table to examine the uni- zontal plane. Moreover, sceneries can magically emerge— verse beyond the cell’s membrane by looking through the protruding from the flat surface of the tabletop, which acts peepholes—very much like looking into a well—and coor- like a membrane—and evolve into a 3D landscape (Color dinate their head movements with the endocytotic process Plate B). to explore the content of the cargo coming into the cell: the material that is enclosed inside the vesicles and that flows The Coalescence of the Virtual and Physical between both sides of the flat surface. Space in the Artwork Endocytosis One thing that must be considered methodologically when Endocytosis and exocytosis, its complementary process, are using the RW is that its horizontal screen is viewed from an both essential biological processes for cell growth and cell-to- unusual angle (compared with vertical displays) and is rela- cell communication and, hence, crucial for the evolution of tively small. Thus, to enhance depth perception, it is essential eukaryotes and the origin of multicellular organisms. Endo- to not only use an adequate projection method but to care- cytosis encompasses the movement of cargo into the cell by fully scale and arrange the objects in the virtual space. Users means of vesicles that emerge from the cell’s membrane and can lean on the tabletop and explore objects and their sur- navigate through the interior of the cell [21]. In the artwork rounding space by simply moving the head and body slightly Endocytosis, I use endocytosis as a metaphor to show the (Fig. 1). This natural and simple interaction, paired with the evolution of a flat land into a 3D world—the conceptual evo- stereoscopic visual stimulus, immerses users in the synthetic lution in the perception of space elegantly depicted by Abbott environment [23,24].

Simó, The Virtual Reality Art Installation Endocytosis 125

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_01265 by guest on 28 September 2021 Hardware and Software the spatial depth, the narrative and the dynamics of the envi- The Virtual Reality Laboratory at the Bauhaus University ronment in harmony with the RW’s physical characteristics. (VRLBU) has an in-house–built RW with an infrared track- Moreover, horizontal stereoscopic displays can be enhanced ing system and active stereo display. This RW is capable of with multitouch interaction that allows users to directly becoming a multiuser stereo system that allows the presenta- manipulate graphics by touching the display. The design of tion of more than one stereoscopic view on a single projec- stereoscopic multitouch interfaces presents the difficult but tion screen [25]. The software system used for this project is intriguing challenge of maintaining a direct interaction with Avango, a programming framework for distributed, interac- objects that have negative parallax without destroying the tive virtual environment applications [26]. stereo effect. Research in this domain [27] could take advan- tage of the human sense of proprioception that allows us to Conclusions and Future Research know our hands’ positions and gestures without looking at The horizontality of the RW’s stereoscopic display is a valu- them. Thus, users could naturally interact with objects float- able feature that can be integrated into the design of the vir- ing above the screen—using a set of hand gestures that can tual environment. Such a property can be used to seamlessly be performed out of sight anywhere on the tabletop surface— merge the virtual space with the physical space by organizing and avoid the problem of occlusion.

Acknowledgments with Its Application to the Fine and Useful Arts and to Education (London: John Murray, 1856). This project was funded by a study grant of the German Academic Ex- change Service (DAAD). I would like to thank Professor Bernd Fröhlich 7 S. Steinman, B. Steinman and R. Garzia, Foundations of Binocular Vi- and the Virtual Reality Systems Group at the Bauhaus University for sion: A Clinical Perspective (New York: McGraw-Hill Medical, 2000). their support and inspiration. The Foundation for Science and Tech- nology (FCT), Portugal, is acknowledged for providing the financial 8 E. Edirisinghe and J. Jiang, “Stereo Imaging, an Emerging Technol- support to write this article and to continue this research (SFRH/ ogy,” Proceedings of SSGRR 2000 International Conference (2000) BSAB/113696/2015). CD-ROM. 9 R. Ferragallo, A Manifesto, Directed to the New of Stereo References and Notes Space in the Visual Arts and the Art of , 12 November 1972. Archived at (accessed 1 C. Wheatstone, “Contributions to the Physiology of Vision. Part the 26 June 2015). First. On Some Remarkable, and Hitherto Unobserved, Phenomena of Binocular Vision,” Philosophical Transactions of the Royal Society 10 C. Hendrix and W. Barfield, “Presence within Virtual Environments of London 128 (1838) pp. 371–394. as a Function of Visual Display Parameters,” Presence: Teleoperators and Virtual Environments 5, No. 3, 274–289 (1996). 2 Binocular vision and rivalry have been described throughout history 11 V. Sorensen and R. Russett, “Computer Stereographics: The Coales- by various authors before Charles Wheatstone. However, Wheat- cence of Virtual Space and Artistic Expression,” Leonardo 32, No. 1, stone is recognized as the first author to articulate that “disparity 41–48 (1999). alone defined the binocular depth perceived” (N.J. Wade, “On the Late Invention of the Stereoscope,” Perception 16, No. 6, 785 [1989]). 12 I. Cho et al., “Evaluating Depth Perception of Volumetric Data in Wheatstone invented the stereoscope, with which he demonstrated Semi-Immersive VR,” Proceedings of the International Working Con- the basis of stereopsis and established the principle of stereoscopy. ference on Advanced Visual Interfaces, ACM (2012) pp. 266–269. For a review of the early history of stereoscopic binocular vision, see N.J. Wade, A Natural History of Vision (Cambridge, MA: MIT Press, 13 O. Grau, Virtual Art: From Illusion to Immersion (Cambridge, MA: 1998) and B. Bowers, Sir Charles Wheatstone FRS: 1802–1875, 2nd Ed. MIT Press, 2003). (London: The Institution of Electrical Engineers in association with the Science Museum, 2001) pp. 45–54. 14 J. Grey, “Human-Computer Interaction in Life Drawing, a - ist’s Perspective,” Proceedings of the 6th International Conference on 3 “. . . the stereoscope would have been limited to flat-sided geometric Information Visualization, IEEE (2002) pp. 761–770. figures, without the invention of the daguerreotype” (M.A. Gaudin, quoted in D. Pellerin, “The Origins and Development of Stereos- 15 A. Simó, “A Horizontal Stereoscopic Projection System for Working copy,” in F. Reynaud, C. Tambrun and K. Timby, eds., Paris in 3D, at the Artist Studio & Mimesis, the Function that Made the Organ,” from Stereoscopy to Virtual Reality [Paris: Booth-Clibborn, 2000] p. SIGGRAPH Sketches and Applications (2003). 44). 16 W. Krueger and B. Froehlich, “The Responsive Workbench,” IEEE Computer Graphics and Applications 14, No. 3, 12–15 (1994). 4 To date, hypotheses about the production of stereoscopic images before Wheatstone have been considered either inconsistent or in- 17 R. Göttig, J. Newton and S. Kaufmann, “A Comparison of 3D Visu- conclusive. For the case of the debate about Chimenti’s , alization Technologies and Their User Interfaces with Data Specific see N.J. Wade, “The Chimenti Controversy,” Perception 32, No. 2, to Architecture,” in J. van Leeuwen and H.J.P. Timmermans, eds., 185–200 (2003). For the case of Leonardo’s Mona Lisa, see C.C. Car- Recent Advances in Design & Decision Support Systems in Architec- bon and V.M. Hesslinger, “Da Vinci’s Mona Lisa Entering the Next ture and Urban Planning (Dordrecht: Kluwer Academic, 2004) pp. Dimension,” Perception 42, No. 8, 887–893 (2003). 99–111. 5 L. Wilcox and J. Harris, “Fundamentals of Stereopsis,” in D. Dartt, 18 M. Dolinsky et al., “Collaborative Virtual Environments Art Exhi- J. Besharse and R. Dana, eds., Encyclopedia of the Eye, Vol. 2 (Oxford: bition,” Stereoscopic Displays and Virtual Reality Systems XII 5664 Elsevier/Academic Press, 2010) p. 171. (2005) pp. 641–652.

6 D. Brewster, The Stereoscope; Its History, Theory and Construction, 19 L.J. Rosenblum et al., “The Virtual Reality Responsive Workbench:

126 Simó, The Virtual Reality Art Installation Endocytosis

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_01265 by guest on 28 September 2021 Applications and Experiences,” Proceedings of the British Computer 25 B. Fröhlich et al., “Implementing Multi-Viewer Stereo Displays,” Society Conference on Virtual Worlds on the WWW, Internet, and Proceedings of WSCG International Conference (2005) pp. 139–146. Networks (1997). 26 J. Springer, B. Froehlich and H. Tramberend, “On Scripting in Dis- 20 L.D. Cutler, B. Fröhlich and P. Hanrahan, “Two-handed Direct Ma- tributed Virtual Environments,” Proceedings of the 4th International nipulation on the Responsive Workbench,” Proceedings of I3D ’97, Immersive Projection Workshop (2000). ACM (1997) pp. 107–114. 27 G. Bruder, F. Steinicke and W. Stuerzlinger, “Touching the Void Re- 21 E.J. Ungewickell and L. Hinrichsen, “Endocytosis: Clathrin-Medi- visited: Analyses of Touch Behavior on and above Tabletop Surfaces,” ated Membrane Budding,” Current Opinion in Cell Biology 19, No. Proceedings of Human-Computer Interaction—INTERACT2013, Part 4, 417–425 (2007). 1 (2013) pp. 278–296. 22 E.A. Abbott and I. Stewart, The Annotated Flatland: A Romance of Many Dimensions (Cambridge, MA: Perseus, 2002). Manuscript received 17 July 2015. 23 Hendrix and Barfield [10]. Águeda Simó has worked in the area of virtual reality since 24 M. Slater, J. McCarthy and F. Maringelli, “The Influence of Body Movement on Subjective Presence in Virtual Environments,” Human 1992. She specializes in interface and interaction design with a Factors: The Journal of the Human Factors and Ergonomics Society stress on natural interaction paradigms and interactive narra- 40, No. 3, 469–477 (1998). tives that reflect multicultural and bioethical issues.

announce m ent

Leonardo Art Science Evening Rendezvous (LASER)

Since 2008, the Leonardo Art Science Evening Rendezvous (LASER) series of lectures and presentations on art, science and technology has provided spaces for progressive thought leaders to come together to form community and explore the intersections of disciplinary thinking. Owing to its success and popularity, LASER has expanded beyond its birthplace in the San Francisco Bay Area, first to the U.S. East Coast, then across the Atlantic to London—the home of the first European LASER—and today continues to expand to new locations nationally and internationally. We thank all of those who have spoken at, participated in or attended LASER events throughout the years. We owe a special thank you to Piero Scaruffi, LASER founder and chair, for his inspiration and continued dedication, and to the growing list of LASER hosts around the world. To follow LASER events, see .

Simó, The Virtual Reality Art Installation Endocytosis 127

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_01265 by guest on 28 September 2021 b plate

color Color Plate B: the virtual reality installation endocytosis: evolving from a flat land into a three-dimensional world

Detail of Endocytosis: circles lying on the 2D surface and becoming spheres when entering the virtual third dimension. Photographs taken off the RW projection screen. (© Águeda Simó) (See article in this issue by Águeda Simó.)

130

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_01265 by guest on 28 September 2021 announc E M ENT

Leonardo Book Series

The arts, sciences and technology are experiencing a period of profound change. Explosive challenges to the institutions and practices of engineering, art-making and scientific research raise urgent questions of ethics, craft and care for the planet and its inhabitants. Unforeseen forms of beauty and understanding are possible, but so too are unexpected risks and threats. The Leonardo Book Series, published by The MIT Press, aims to consider these opportunities, changes and challenges in books that are both timely and of enduring value.

Proposals that address these challenges in terms of theory, research and practice, education, historical scholarship, discipline summaries and experimental texts will be considered. Single-authored books are particularly encouraged.

Full book proposal submission guidelines: .

Inquiries and proposals should be submitted to both:

Leonardo Book Series Doug Sery c/o Leonardo MIT Press Books 1440 Broadway, Suite 422 One Rogers Street Oakland, CA 94612 Cambridge, MA 02142 U.S.A. U.S.A.

Email: .

For a list of Leonardo Book Series titles, visit .

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/LEON_a_01265 by guest on 28 September 2021