Special Issue on Arts, Aesthetics, and Performance in Guest Editors’ Introduction: Homo Ludens in Virtual Environments

1. Introduction (gestural interaction, tangible user interfaces) than at any previous time. In this line, , which provides ‘‘The fate of our times is characterized by rationaliza- presence and immersion, is becoming more important tion and intellectualization and, above all, by the disen- for embodied interactions. Scientists and technologists chantment of the world.’’—Max Weber, 1904/1905 can learn interaction techniques and strategies from body expression experts—artists. Meanwhile, virtual Weber diagnosed the previous era rather precisely. reality can provide an integrative, dynamic platform for People worked hard with a given vocation. Play was sep- arts and performances: a living synthesis of what German arated from work and people were not likely to play any composer Richard Wagner once dreamed in his vision of more. The world was disenchanted and Homo Ludens Gesamtkunstwerk—a comprehensive work of art. We 1 evolved into Homo Sapiens. The magic circle disap- hope this special issue can serve as a good step towards peared. However, this magic circle is recently returning that goal. to our lives through emerging technologies. People enjoy hunting monsters in their garden (e.g., Poke´mon 2. Aesthetic Computing Go) and children paint in 3D space (e.g., Tilt Brush). People’s imaginations are encouraged by the technolo- We encourage our readers to view this special issue gies and play might be restored to our lives. from the perspective of Aesthetic Computing, which is The relationships among art, technology, and science defined as ‘‘the application of art theory and practice to are much stronger today than may be immediately appa- computing’’ (Shem-Shaul et al., 2003). When we have rent. The connections between artists and scientists have an approach encouraging a more cultural, personal and waxed and waned in the eighteenth and nineteenth cen- customized set of aesthetics in computing, we can turies. However, they have been reintegrated in contem- explore more creative and innovative media for software porary art (Packer & Jordan, 2001). Considering that and mathematical structures; make computing more the original meaning of aesthetics is the study of our per- accessible to diverse populations so that they can under- ception of the entire environment (not just the study of stand the concept of computing and utilize it more read- an ‘‘object of beauty’’) (Bolter, Engberg, & MacIntyre, ily; and promote personalization and customization of 2013), we are likely to acknowledge that technological computing structures. We believe that these benefits advances have accelerated the advent of new aesthetics. cause a chain reaction. The application of aesthetics to Thanks to new technologies, we can expand the percep- computing has brought about new media for computing tual experiences of our existing senses and can even cre- and mathematical representations. The new media or ate novel perceptual dimensions that have never been representations of the computing process and its struc- imagined—new presence. Given the paradigm shift from ture play an important role to process, encode, and cognitivism into embodiment, the human body now understand the information. The evolution of the em- has more opportunity for representation in computing bodiment of computing from command line interfaces, to graphic user interfaces, and to tangible user interfaces 1. The space in which the normal rules and reality of the world are proves that such new media make computing more suspended and replaced by the artificial reality of a game world. accessible. Consequently, this pervasiveness of computing Presence, Vol. 26, No. 2, Spring 2017, iii–vii doi:10.1162/PRES_e_00295 resulted in easy personalization and customization and ª 2017 by the Massachusetts Institute of Technology thus, computing can ultimately be used in our everyday

iii

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_e_00295 by guest on 01 October 2021 iv PRESENCE: VOLUME 26, NUMBER 2

lives for ‘‘making special’’—fundamental meaning of art provides an opportunity to integrate physical and digital (Dissanayake, 2003). worlds, which was previously unavailable. This creates Based on this aesthetic computing approach, on the malleable materials and experiences, whether we call it one hand, computer scientists obtain more options in virtual reality, , or . Of aesthetics accompanied by enhanced comprehension and course, it would not be strange to replace ‘‘users’’ with motivation. On the other hand, artists benefit from com- ‘‘artists’’ in all these statements. putational thinking and its underlying mathematical Virtual environments have a reciprocal relationship structures (Fishwick, 2006). In other words, from these with full embodiment. Virtual environments with rich reciprocal interactions between computing and aes- sensory information offer an opportunity for users to be thetics, computing can be enriched by aesthetic theory immersed in the task and context so they explore the and practice while simultaneously facilitating the forma- environment using their entire body as if the situation is tion of new art and aesthetics by providing novel media real. At the same time, their embodied interaction with and platforms for art. the virtual environment system will again increase feel- ings of presence by stimulating their motor areas as well 3. Embodiment and Virtual Environments as sensory areas in their brain. Acting in the virtual envi- ronment beyond simply seeing the computer display will As discussed above, aesthetic computing is deeply certainly boost artists’ and users’ immersion into a new related to embodiment. By complementing or replacing virtual space. Also, it will allow them to engage more the traditional cognitivism, embodied cognition has rap- with their tasks or activities. idly emerged as a new theoretical paradigm in cognitive sciences (see the special issue in TOPICS; Davis & Markman, 2012). This has also brought up a new para- 4. Presence and Telepresence digm, embodied interaction, in Human-Computer Interaction (HCI) (see the special issue in TOCHI; Traditionally, literature has used a space metaphor Marshall, Antle, Van Den Hoven, & Rogers, 2013). about presence, such as being there (Minsky, 1980) or Theories of embodiment pose new topics that designers feeling of being present in an environment other than one and researchers can consider in their interaction design the person is actually in (Sheridan, 1992). One of the (Klemmer, Hartmann, & Takayama, 2006). (1) Users critical values of a virtual environment is that it can not can learn through doing. They think by gesture and only provide the space for representation of reality, movement and identify implicit constraints and prob- which is one of the original goals of art, but also provide lems easily. (2) Users can act through an artifact, rather the tweaked version of reality. This opens up the new than act on it. They perceive the artifact as an extension possibilities for art works. As drawings of romanticism of their body, rather than an independent object. This mimicked neo-classicism, contemporary art is mimicking explains why emotions and affect are getting more im- the work of romanticism but also tweaking it (e.g., ‘‘Le portant in interaction design (Jeon, 2017). (3) Users can De´jeuner sur L’herbe’’ by E´ dourard Manet in 1863 vs. easily perceive the status and response of other users as ‘‘Le De´jeuner sur L’herbe, Les Trois Femmes Noires’’ the distributed cognition theory (Hollan, Hutchins, & by Mickalene Thomas in 2010; as Manet untraditionally Kirsh, 2000) suggests. It facilitates learning by partici- had a lady sit together with men on the grass and stare pating in a community of practice and enhances coordi- forward, Mickalene also had women sit together on the nation based on peripheral participation. (4) Embodi- grass instead of men). In the virtual environment, for ment allows users to prepare for risk and surprise. They example, this tweaking task can be more flexible, and the can pay more attention to their activities and be respon- tweaked version and the original can even be presented sible for their decision making because the consequences at one time or superimposed to compare and contrast of their decisions are more visible. (5) Embodiment (Jeon, Landry, Ryan, & Walker, 2014).

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_e_00295 by guest on 01 October 2021 v

This type of activity can be further expanded by virtual communication) and full interactivity depends on the presence. Jeffrey Shaw once described interactive arts as nature of the communication responses. Both reactive a virtual space of images, sounds, and texts (Kwastek, and fully interactive communications require that com- 2015). Inherently, implies virtual pres- municants respond to each other. However, with quasi ence. The advancement of network technologies enabled interactivity, the content of response may have a reaction virtual presence or telepresence. Since futurists champ- to previous messages, whereas full interactivity acknowl- ioned radio as a new art format in their manifesto in edges prior responses. In other words, to contain full 1933, artists have consistently employed many other net- interactivity, responses should incorporate references to work media for their arts, including satellites, fax the content already exchanged and conjure up memora- machines, BTX systems, mailboxes, the Internet, and ble interactive exchanges. Therefore, we want to empha- mobile data networks (Kwastek). This telepresence also size interpretation and recurrence in interaction because enabled real-time cooperation among artists. For exam- without this concept, any vending machine can be con- ple, even though music is a very time-sensitive genre of sidered an interactive system. art, networked music can be played in different locations at the same time. This type of work goes beyond the tra- 6. Summary of Contributions ditional meaning of ‘‘transmitting information’’ in arts and interaction happens remotely. The manuscripts submitted to this special issue have undergone a rigorous peer-review process in which 5. Interaction and Interactivity each manuscript was reviewed by at least two independ- ent reviewers. In total, 12 manuscripts were submitted Another core concept of this special issue is inter- to the special issue. Guest editors performed meta- action. Like other terms, the concept of interaction has reviews on the papers in each round of reviews (2–3 included many different meanings and has consistently rounds of review), and finally the five best articles were evolved. For instance, behavioral psychologists focused selected for publication. The articles of this special issue on the stimulus–response relationship as a reaction in incorporated and elaborated upon some of the critical our bodies and sociologists looked at a communicative concepts we have discussed. process, where people share their experience. Norbert Carlson, Sun, Cuykendall, Lantin, Schiphorst, and Wiener (1950), in his cybernetics, extended this con- Corness described the project done by collaboration struct by varying the types of feedback and by including between researchers and artists in the article ‘‘Beyond reflexive reactions and systems that are capable of learn- the Here and Now: Exploring Threaded Presence in ing. This implies that complex dynamic systems can Mediated, Improvised Performance.’’ The authors include interpretation, which mediates between actions explored the concept of presence by examining perform- and reactions. ers’ experiences in an improvisational, telematics per- Kwastek (2015) listed the fundamental features of formance and coined the term, ‘‘Threaded Presence,’’ interaction as real-time exchange and presence; control which bridges the gap between ‘‘situated’’ and and feedback; and selection and interpretation processes. ‘‘extended’’ presence. They described how technology However, she acknowledged that not all these are neces- can contribute to the entire experience of performance sary components to make up the ‘‘interactive process.’’ by serving as another performer. For example, the concept of real-time exchange is not Rumori and Marentakis explored the potential of spa- requirement and also more passive interaction is possible tial sound technology in the intersection of artistic rather than active control. investigation and scholarly research in the article Rafaeli (1988) refined the concept of full interactivity. ‘‘Parisflaˆneur. Artistic Approaches to Binaural Technol- He has suggested that a distinction between quasi inter- ogy and Their Evaluation.’’ The authors provided the activity (e.g., two-way communication or reactive case study and its iterative improvements through the

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_e_00295 by guest on 01 October 2021 vi PRESENCE: VOLUME 26, NUMBER 2

usability test. They showed that the full potential of have witnessed how virtual environments can expand immersive media can be achieved only by investigating modalities of computing and art in this special issue. Aes- technology within the context of aesthetic experience. thetic qualities, such as mimesis, symmetry, complexity, The next two articles are dedicated to creating inter- parsimony, minimalism, and beauty, have been trans- active artificial agents. In the article ‘‘Flow Fields and ferred to/combined with computing. These aesthetic Agents for Immersive Interaction in Mutator VR: qualities will be further enhanced when integrated with Vortex,’’ Putnam, Latham, and Todd discuss the chal- more diverse, culture-specific approaches. Various tradi- lenges in creating Mutator VR: Vortex, a virtual reality tions and philosophies in different cultures can be experience based on interaction with semi-autonomous applied to aesthetic computing by adding new frame- agents. The authors took aesthetic inspiration from or- works of both implementation and analysis. This attempt ganic forms found in the natural world. To this end, will simultaneously make both computing and art more the authors selected curved-based synthesis and flow accessible to each person. grains as mathematical constructs to provide a natural We very much appreciate all the authors and reviewers integration of modalities, which resulted in a sense for their contributions to making this special issue. We of agent autonomy and playful and emergent hope that you can enjoy this magic circle and reclaim interactions. ‘‘Homo Ludens’’ for a moment. Ch’ng, Harrison, and Moore also made a mixed-real- ity art installation of artificial life of agents, in the article ‘‘Any sufficiently advanced technology is indistinguish- ‘‘Shift-Life Interactive Art: Mixed-Reality Artificial Eco- able from magic.’’—Arthur Clarke, 1973 system Simulation.’’ Their artistic focus was to explore the parallels between the human thinking process and Myounghoon ‘‘Philart’’ Jeon computer process, both of which are non-linear systems. Michigan Technological University, USA The authors explored further people’s perceptions and mind states when they shift their attention between the Paul Fishwick and the real world. They merged the physi- The University of Texas at Dallas, USA cal and virtual world as the mixed reality system and inte- Guest Editors grated artificial life, ecology, physical sensors and partici- pants’ interaction. A more ecological approach is described in the last pa- per, ‘‘The Networked Sensory Landscape: Capturing and References Experiencing Ecological Change Across Scales,’’ by May- ton, Dublon, Russell, Lynch, Haddad, Ramasubrama- Bolter, J. D., Engberg, M., & MacIntyre, B. (2013). Media nian, Duhart, Davenport, and Paradiso. The authors studies, mobile augmented reality, and interaction design. explored how ubiquitous sensing can make the natural Interaction, 20(1), 36–45. environment artistic and interpretable, which can facili- Davis, J. I., & Markman, A. B. (2012). The future of embodied tate our understanding and experience of ecology. In cognition. Topics in Cognitive Science, 4(4), 685–793. Dissanayake, E. (2003). Retrospective on homo aestheticus. their long-term research, they integrated multitude com- Journal of the Canadian Association for Curriculum Studies, ponents, including sensor hardware, web services, public 1(2), 7–11. user interfaces, and UAV, VR/AR, and wearable devices. Fishwick, P. A. (Ed.) (2006). Aesthetic computing. Cambridge, MA: The MIT Press. 7. Conclusions Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cog- nition: toward a new foundation for human-computer inter- Fishwick (2006) proposed three broad topics in action research. ACM Transactions on Computer-Human aesthetic computing: modality, quality, and culture. We Interaction (TOCHI), 7(2), 174–196.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_e_00295 by guest on 01 October 2021 vii

Jeon, M. (Ed.) (2017). Emotions and Affect in Human Factors Minsky, M. (1980, June). Telepresence. Omni, pp. 45–51. and Human-Computer Interaction. San Diego, CA: Packer, R., & Jordan, K. (2001). Multimedia: From Academic Press. Wagner to Virtual Reality. New York: W. W. Norton & Jeon, M., Landry, S., Ryan, J. D., & Walker, J. W. (2014). Company. Technologies expand aesthetic dimensions: Visualization Rafaeli, S. (1988). Interactivity: From new media to communi- and sonification of embodied Penwald drawings. Proceedings cation. Sage Annual Review of Communication Research: of the International Conference on Arts and Technology. Advancing Communication Science, 16, 110–134. Beverly Klemmer, S. R., Hartmann, B., & Takayama, L. (2006). How Hills, CA: Sage. bodies matter: Five themes for interaction design. Proceed- Shem-Shaul, N. B., Bertelsen, O. W., Bolter, J., Bruns, W., ings of the 6th Conference on Designing Interactive Systems. Bureaud, A., Diehl, S., . . . Entacher, K. (2003). Aesthetic Kwastek, K. (2015). Aesthetics of interaction in digital art. computing manifesto. Cambridge, MA: The MIT Press. Sheridan, T. B. (1992). Musings on telepresence and virtual Marshall, P., Antle, A., Van Den Hoven, E., & Rogers, Y. presence. Presence: Teleoperators and Virtual Environments, (2013). Special issue on the theory and practice of embodied 1(1), 120–126. interaction in HCI and interaction design. ACM Transac- Wiener, N. (1950). The human use of human beings: Cybernetics tions on Computer-Human Interaction (TOCHI), 20(1). and society. Da Capo Press.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/PRES_e_00295 by guest on 01 October 2021