Postdigital Art - Proceedings of the 3Rd Computer Art Congress Edited by K

Home , Chaos gods, Digital poetry, FIGlet, Maurice Benayoun, ZX81 character set

PostDigital Art - Proceedings of the 3rd Computer Art Congress Edited by K. Zreik, R. Gareus November 2012 http://postdigital.eu

Published by europia Productions 15, avenue de Segur´ 75007 Paris, France Tel: +31 1 45 51 26 07 Email: [email protected] http://www.europia.fr

Credits Cover design: Khaldoun Zreik Logo: Robin Gareus Layout: Robin Gareus Typesetting: LATEX and pdfTEX

Printed in the EU ISBN 979-10-90094-12-3 c 2012 europia productions

Tous droits reserv´ es.´ La reproduction de tout ou partie de cet ouvrage sur un support quel qu’il soit est formellement interdite sauf autorisation expresse de l’editeur´ : Europia Productions. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the publisher Europia Productions

Partners and Sponsors

CiTu, Laboratoire Paragraphe 104 Centquatre, Paris

Europia Conseil regional ˆIle de France

cap-digital digitalarti - Digital Art International

Arts and Humanities Department, Design Research Foundation the Open University of Catalonia

Le Cube

iv Foreword

The third Computer Art Congress (CAC.3) is dedicated to PostDigital Art. It is a making in many senses and invites artists, intellectuals, scientists and technologists to share their imaginations, creations, inventions and visions of the post digital art. CAC.3 observes that the world has never appropriated any technology in the same manner than the digital. This technology has penetrated and dominated almost all facets of our everyday life. It has had, obviously, important impacts on our culture, economy, society, . . . and cognition. We believe that our ways of perception, interpretation and reasoning have not been same before and after having dealt with the digital world. Whats more digital technology has become more than part of our life, it has nearly become transparent. Nicholas Negroponte declared the digital revolution over in 19981: “like air and drinking water,” digital would be noticed only by its absence, not its presence. Simon Jenkins recalled this point2 “Don’t tell me you are still putting e- and i- in front of your product or talking ‘platforms’, like some naughtiest nerd. That is so yesterday”, and he persisted that “Post-digital is not anti-digital. It extends digital into the beyond. The web becomes not a destination in itself but a route map to somewhere real”. The term Post Digital has recently come into use in the discourse of digital artistic prac- tice3. The term aims to call attention to “an attitude that is more concerned with being human, than with being digital”3. Roy Ascott considers distinction between digital and ”postdigital” is part of the economy of reality3. For Mel Alexenberg4, postdigital as ad- jective, addresses the “humanization of digital technologies”. About PostDigital Art, Adam Tinworth5 points out tow important facts : - “Theres a rule of thumb in the real estate business that if you want to know which part of a city is going to go up-market next, look at where the artists go to work.” - “Everything we do is inﬂuenced by digital technology. Just as air and water, the property of being digital is only noticed when it is not there, not when it is there.” For all those reasons, and in order to preserve the artistic (and humanistic) part of the computer art, the advisory board of CAC retained, during the last congress in Mexico (in 2008), the Post Digital Art as the main topic for CAC.3. In that sense, CAC.3 considers PostDigital Art as an open creative way to draw out the evolving of our relation to information and communication technology as a dominant in the globalization paradigm we are living. PostDigital Art experience has to be considered as intellectual therapy that challenge actors of the society to rethink their innovation approaches and the way they perceive the world, to explore new dimensions of our space, to go forward, to trace their own path, to be followed CAC.3 count on the abilities of artists to explore digital and extra digital spaces in order to anticipate new technological issues that can inﬂuence our post digital world.

1Teressa Iezzi & Ann-Christine Diaz, May/24/2010, “Are we Post-Digital Yet?”, http://creativity-online.com/news/are-we-postdigital-yet/144055 2Simon Jenkins, Dec/1/2011, The Guardian, http://www.guardian.co.uk/commentisfree/2011/dec/01/post-digital-world-web 3Wikipedia contributors, “Postdigital” Wikipedia, The Free Encyclopedia (accessed October/27/2012), https://en.wikipedia.org/w/index.php?title=Postdigital&oldid=483815596 4Mel Alexenberg, 2011, The Future of Art in Postdigital Age, Intellect Books 5Adam Tinworth, Jan/2012, “Can a new culture grow from Post Digital art?”, http://nextberlin.eu/2012/01/can-an-new-culture-grow-from-post-digital-art/

Conference Organization

Khaldoun Zreik Robin Gareus

Technical Team

Centquatre CiTu

Conference Design and Website

Robin Gareus

Special Thanks to

Imad Saleh Nada Karami Marion Chapel Martin Colomer-Diez Maurice Benayoun Sandrine Marsaudon Valerie Senghor

...and to everyone else who helped in numerous places after the editorial deadline of this publication.

vii Review Committee Alan Dunning Alberta College of Art + Design, University of Calgary, Alberta, Canada Alok b. Nandi Architempo Annick Bureaud Independent Curator Charlie Gere Lancester Institute for contemporary arts Christian Jacquemin LIMSI, CNRS Everardo Reyes Universite´ Paris 13 Frank Dufour Arts and Technology Program, The University of Texas at Dallas Iliana Hernandez´ Pontiﬁcia Universidad Javeriana, Bogota´ Jack Stenner Art + Technology Program, University of Florida Jeffrey Shaw School of Creative Media, City University Hong Kong Joseph Nechvatal School of Visual Arts, New York Khaldoun Zreik CITU-Paragraphe (University Paris 8) Lev Manovich University of California, San Diego Luisa Paraguai Nuova Accademia di Belle Arti, Milano, Italy Malu Fragoso Federal University of Rio de Janeiro Marc Veyrat Poleˆ Image & Information (Laboratoire IREGE - IAE Savoie Mont-Blanc) Maurice Benayoun CiTu - Universite´ Paris 8, School of Creative Media, City University Hong Kong Mirko Tobias Schaefer Utrecht University Nils Aziosmanoff le Cube Pau Alsina Arts and Humanities Department, Universitat Oberta de Catalunya Paul Woodrow University of Calgary, Alberta, Canada Paul Klein San Francisco Art Institute Paul Magee Factulty of Arts and Design, University of Canberra, Australia Pierre Boulanger University of Alberta, Canada Reza Beheshti Design Research Foundation, Netherlands Ricardo Mbarkho University of Balamand Richard Coyne Edinburgh College of Art, The University of Edinburgh Robert Woodbury School of Interactive Arts and Technology - Simon Fraser University Robin Gareus linuxaudio.org, Universite´ Paris 8 Ron Burnett Emily Carr University of Art and Design Roy Ascott University of Plymouth, UK Safwan Chendeb CiTu - Universite´ Paris 8 Stuart Moulthrop Yale Gordon College of Liberal Arts, University of Baltimore Tania Fraga IMA-SP Vincent Boyer LIASD - Universite´ Paris 8

viii Table of Contents

The search for emergence in New Media Art practices 1 • Pau Alsina

Brief Reﬂection on the Anonymity 7 • Marcos Salazar Delﬁno

Future Potentials for ASCII art 13 • A. Bill Miller, Anders Carlsson

Hyper-Production and the Value of Exquisite Corps on the Web 25 • Blanka Earhart

21st Century Brazilian Computer (Experimental) Art 31 • Tania Fraga, Malu Fragoso

The Post Digital Art is made of Paper, Cardboard and ABS 43 • Filipe Pais

On the separation between the esthetic and the functional - And how the digital • realm will steal form 51 Aurelien´ Michon, Cedric Flazinski, Clement´ Chalubert

From Paper to Portable Devices: How to script and read comics? 59 • Vincent Boyer, Jer´ emy´ Raulet, Catherine Sauvaget

Designing Natural User Interfaces with Depth Sensing Technologies like the Kinect • Sensor - A Tracking Framework for Artists and Designers 69 Michaela Honauer, Jens Geelhaar

New technologies and non technical students: making contact with creative digi- • tal writing 75 Gaetan´ Darquie´

Disrupting 3D models 91 • Everardo Reyes

Digital Preservation of Ju Ming Stone Sculpture in Taiwan 97 • Naai-Jung Shih

U-rss and the dark side of the moon 103 • Marc Veyrat, Franck Soudan

Connected creation: The Art of Sharing 115 • Florent Di Bartolo

Cinema Beings 121 • Alain Lioret

Facebook Chronicles 129 • Regina Freyman

ix Architectural Ornamentation and Fabrication with Multi Agent System 137 • Subhajit Das, Florina Dutt

Autonomous social Avatars (AsA) 145 • Juan Pablo Bertuzzi, Safwan Chendeb, Khaldoun Zreik

Digital Environment to Envision and Experience the Art of Light and Space 151 • Cheng-Hsiu Chuang, Nan-Ching Tai

Art image classiﬁcation using Bag-of-Visualterms representation 157 • Ferran Reverter, Pilar Rosado, Miquel Angel Planas, Eva Figueras

Post Anxiety Art: Economies and cultures of digital painting 161 • Jo Briggs, Mark Blythe

x The search for emergence in New Media Art practices

Abstract discussed by physicist Doyne Farmer, to whom emergence "is not magic, but it feels like magic."2 As a follow up of the long pursued ideal of Art as Life, or better, as Creation of the Creation, in this paper we intend to explore the use of 2 Definitions of Emergence and its Emergent dynamics in Art practices linked to relevance in Art. Science and Technology. We articulate this research through the analysis of those significant Although there are several definitions of what may fields of New Media art practices that have be recognized as emergence, one of the more approached to the emergent properties as a key accepted is the one proposed by Jeffrey Goldstein force field in their development. in the inaugural edition of magazine Emergence. For Goldstein, "emergence happens at the roots of the appearance of new and coherent structures, Keywords patterns and properties during the self- emergence, complexity, new media art, digital organization process in complex systems. The art, a-life art. common features are: 1 - radical novelty (characteristics that have not been previously observed in systems); 2 - consistency and 1 Introduction correlation (meaning integrated "wholes" that keep The concept of emergence has a long history, intact during a period of time), 3 - a macro or during which it has acquired different meanings in global level (with some quality of "totality"), 4 - the several areas of knowledge where it occurs. If being the product of a dynamic process (it nowadays different theoreticians speak about develops), and 5 - being "ostensible", that is to say, complexity as a paradigm of the new millennium, it can be noticed.”3 emergence seems to become the explanation of how complexity is evolving. Complexity is Although that definition may be widely accepted considered an emergent phenomenon, and in the scientific community, it doesn't include all emergence, which is produced by self-organizing the different shades and definitions associated systems, is considered a way of understanding with the same term, because, as Goldstein himself natural phenomena such as hurricanes, ecosystems says, "emergence works not so much as an and complex organisms as human beings and life explanation, but rather as a descriptive term that itself, to mention a few examples. points toward patterns, structures or properties that are exhibited in a macro scale"4. Although Certainly, the concept of emergence became a very there is no universal consent about its definition, inspiring and quite controversial term, in which we may explain it, recognize it and measure it reductionist postures live together, such as through its behaviors. We can recognize an Bertrand Russell's, to whom the emergent qualities emergent behavior when it is a complex behavior are just epiphenomena without any scientific that happens as a result of a construction "bottom meaning, since "through analysis we may arrive to 2 a structure in which the properties of the complex WALDROP, M.M. (1992). Complexity: The Emerging Science at the Edge of Order and Chaos. New York: Touchstone Simon & can be inferred from those that come from the Schuster parts," 1with other positions, such as those 3GOLDSTEIN, J. (1999). “Emergence as a Construct: History and Issues”. Emergence. Vol. 11, 1999, pp. 49-72

1 RUSSELL, B. (1927). The Analysis of Matter. London: Allen & 4GOLDSTEIN, J. (1999). “Emergence as a Construct: History Unwin. pp. 285-286 and Issues”. Emergence. Vol. 11, 1999, pp. 46

1 up", produced from a series of simple behaviors 3 Eigth Steps towards Emergence in Art (for example, simple rules). Therefore, we may say practices linked to Technoscience. that emergent properties are properties of the global system that appear from the complex Even though there is a long history of interaction among its parts, that is to say, they refer intersections between art, science and technology to the properties or processes of a non-reducible in History, it is true that since the development of system to the properties or processes of its digital technologies, (and moreover the whole constituent parts, obtaining, in many cases, totally infornation and communication technologies) unexpected results and hardly deducible from changes and transversality processes between them, as well as their local interactions. different fields have been considerably accelerated . ICTs are everywhere and act between Thus, we would say that the emergence concept is fields connecting bridges and constructing easier not a product of an organized, rigorous unique ways of collaboration and transdisciplinarity theory, "but a set of ideas that have in common the works. notion that within dynamic patterns it may exist These digital technologies have pushed forward an inscribed simplicity that is capable of being the New Media Art practices, including new partly discovered through analytic, logical and technologies and new sciences as key contextual conceptual developments5. The diversity of discourse, techniques or materials to be based on. theories of emergence and their possible And this has occured in parallel with the study of applications are enormous and, therefore, difficult emergent properties , that have been progressively to synthesize, but we could highlight some gaining relevance in contemporary technosicence features that are common to the different postures and culture due their relevance in complexity relating to emergence. These emergent properties, systems. on the other hand, are also explored in the scope of the artistic practice linked to techno-sciences, We have focused In our research in the crossing where this exploration seeks in some way the between emergence and new media art practices materialization of the utopian ideal of fusion through what we consider are the different key between art and life, and it is connected with the steps towards its pursue. We have then divided it issue of creation itself. in eigth steps that, with different approaches and types of technologies and art practices, have In this sense, the theory of complexity models approached to the study of emergent propperties. the material systems using techniques proper of non-lineal dynamics, through the demonstration of 3.1 Life and Emergence in Bioart. the topologic features of diversity (the distribution of singularities) that affect the series of trajectories We should then start saying that in relation to in phasic space, revealing patterns (indicated by life, emergence itself has been the underlying cause the atractors in models), thresholds and the of the evolution of emergent phenomena in required intensity of those systems' triggers (events biological evolution, since they are the synergies that move systems toward thresholds that activate produced by organized systems, which allows to patterns).6 Thus, showing the spontaneous articulate the same emergence later on. A appearance of indicators of patterns and modification in any of the parts may affect the thresholds in models of complex systems' synergy produced by the whole, for better or for behavior, the theory of complexity enables us to worse. A mutation associated to any feature can be think material systems in terms of their potential of "the difference that makes the difference," said immanent self-organization.7 Bateson. From a synergistic perspective, the functional effects produced by the wholes have much to do with the explanations of the parts. But once life is reduced to gene by the discourse of Biotechnologies ,then, in the context of a "geneticized" life, the part, the gene, designates the whole, life, and the emergent dynamics as 5LISSACK M. R. (1999). “Complexity: The Science, its Vocabulary, and its Relation to Organizations”. Emergence. Vol explanatory cause of life itself is reduced to the 11: 1999, pág 112 control of the information coded in the gene, 6 PROTEVI, J.; BONTA, M. (2004). Deleuze and Geophilosophy: devoid of context. For example these are some of A Guide and Glossary. Edinburgo: Edinburg University Press. the main critiques inscribed in BioArt practices, such as Critical Art Ensemble or artists Oron Catts 7PROTEVI, J. (2006). “Deleuze, Guattari and Emergence”. En: Paragraph. 29:2, pág 19-39 and Ionatt Zur from Symbiotica, dealing with alife

2 materials and the biotech discourses that are escapes from the design of human computational constructing an hegemonic narration of life. models, and finally turns into something uncontrollable, with some structures that are not restricted by stable knowledge, formal 3.2 Robot Art and the Body-Mind relationships or causalities. Because the restriction emergence. of the technological frame in which one tries to As to the role of the body, we may observe that reproduce emergence as a constituent of life in present-day theories it becomes the basis of hampers the creation of emergence, while trying to cognition, that in its turn becomes the process of formalize emergence itself. At this point, it should life. The enactive conception of the organism be pointed out how the cultural dynamics of art proposes a mind inextricably linked to the body, are a much more viable substratum for emergence, now understood as embodied mind, in which enabling the art objects to become open, emergent perception is not just activated in response, but and unpredictable. Ironically, emergence in the elapses from the action on the environment, as artistic practices with artificial life does not reside movement.8 Cognitive structures emerge from so much in simulations, as in the form in which recurrent sensitive patterns, and the organism those artistic practices modify what we think and becomes the construction of a certain selection of feel about the world. An example of this virtual multiplicity of what the body might computational emergence can be explored in the become. That enactive cognition represents a story work LifeWriter (2006), by Christa Sommerer and of the structural coupling of the body that makes a Laurent Mignonneau, explores the methaphor of scripture as a generator of life, through the emerging world to emerge, and it acts through a network behavior proper of the artificial life technologies used in formed by several levels of interconnected the piece. Or another example can be found in José sensorial-motor subnets. And thus the mental Manuel Berenguer explores the computational contents draw in their own organization -- self- emergence in Luci (2008), an installation inspired by the organization -- a sensible noticed world that is bioluminescent behavior of Kuala Selangor's fireflies. partly an emergence, an autopoietic creation that proceeds from the ordering in classes of those same mental contents. Thus, reality-world and 3.4 The emergence of/in Artificial mental phenomena would meet in a continuous, Intelligence and Art. transforming dialogue, and this new model would Regarding the different theories of cognition that demand a re-evaluation of developments in the account for the different approaches on Artificial field of robotics, by means of trying to create an Intelligence, we can observe the evolution from the embodied artificial intelligence, that emerges from early theories of information processing to the interaction with the environment and with the connectionism and the emergent theories of machine's own materiality, thus configuring new artificial intelligence. Based on the attempt of adaptive computer devices. For example Robots by simulating the brain's natural processes by the researcher Luc Steels, from SONY Lab Paris and Vrige Universiteit Brussel (Al Lab), explore the emerging connectionists, one fell in the romantic ideal of creation of language, while Rodney Brooks tries to focus equality between mind and machine. Knowing on artificial intelligence based on the robots emerging through experience, rather than through received behavior. instructions, gave place to the formation of artificial neural networks capable of learning and 3.3 Computational Emergence in A-life providing feedback to the system, establishing Art. connections and values suited to its elements. Later Regarding emergence in the context of artificial on, the progresses of emergent artificial life, we see how in the artistic practices that use intelligence have linked, in an implicit and explicit those technologies, emergence and complexity are way, computers to the world of the humans, constantly evoked, with unexpected results through all kinds of biologic and social metaphors. ascending from a pre-designed technological The strong program of research in Artificial substratum. But this technological pre- design is Intelligence gave way to the weak program, exactly what confers to it a differentiated status of simulation gave way to emulation and "computational emergence," which, nevertheless, constructionism, which in a pragmatic way uses we could say is not an authentic emergence, systems of diffuse logic, artificial neural networks, because it is restricted to the technological parallel computing and quantum computing to computational model itself. Hence, artificial life make a world experience to emerge in a computational way. Some artists as Ken Feingold 8VARELA, F.; THOMPSON, E.; ROSCH, E. (1991). The Embodied Mind. Cambridge: MIT Press. or David Rokeby have been exploring the

3 possibilities of AI and the emergence of indeed to the object, but are not materials, in communication patterns through their artworks. In principle. The unit, for example, is a predicate of Rokeby's (n)chant a community of entities linked objects, but is not a material property, and by a network. intercommunicate, and through although the unit may be considered as one of the doing so, 'synchronize' their individual internal transcendental categories of knowledge, we could 'states of mind'. As Rokeby explains ”When left also consider it immanently as a virtuality, that is, uninterrupted to communicate amongth belonging to matter itself and not originating from themselves, they eventually fall into chanting, a the exterior by a transcendental understanding. In shared stream of verbal association. This virtuality, the same emergent origin produces consensus unfolds very organically. The systems different forms not contemplated by potency: soap feel their way towards each other, finding bubbles, crystals, embryogenesis, migratory resonance in synonyms and similar sounding words, working through different formulations of movements, economic transactions, and so on. similar statements until finally achieving unison.” 3.7 The Digital in Art.

Regarding the digital, present in the relationship 3.5 Software art, Calculability and between art and computer science, we can analyze emergence the different attributes that characterize it, the epistemological and ontological shifts that bring Relating to calculability and programmability, about those new ways of proceeding with inscribed in the software and the programming information processing, that emerge structurally languages, these gain relevance while they devoid of the context that receives it, providing the construct ways of seeing, knowing and making the processing of images, sounds and texts of new world, which in their turn contain a model of that properties and, therefore, of new possibilities that, part of the world to which they belong, and which in the context of art and of engineering, have been they shape each time they are used. We see the progressively explored. Under the enclave of the Cartesian metaphors that have articulated their theory of information and later on of cybernetics, a evolution assigning categorizations, where the scope of knowledge is structured that departs from software comes to be considered an abstraction of a series of presuppositions that model a particular the hardware, or even the hardware becomes a idea of mind and that, somehow, explore the ideal metaphor in itself, when algorithms may work on reach of a computational mind. Starting from those any imaginable material. A Cartesian dualism that information devoid of context, properties as the separates body and mind, and that, if it were fact that it is converted to numeric representation, abandoned, might abandon the supposition that that it may be modulated, automated, variable and the software is immaterial and the hardware, transcodifiable prosper. Those properties have material, and start considering the software as the revealed fundamental for the understanding of the material inscribed in the coded and stored development of computer technologies, that have algorithm, in one more step towards the been trying to approach emergent phenomena. In materialism proper of the emergent dynamics this sense for example we should visit Canvi & approached. Temps (2009), of Bestiario and Arts Santa Mònica, are two navigation spaces as much historical as traverse, 3.6 The Virtual, the Real and the Actual. offering a wide and relational vision of the strategies, Regarding the concept of virtual, which we may methodologies and topics of interest that were developed in the science of complexity field. approach through virtual-reality technologies and their artistic appropriation, we might expose how it seemingly articulates as an oxymoron, while 3.8 Art, Networks and emergent properties intending to effect the programming of a complete In the case of networks, one should keep in mind simulation of reality. But that relationship between their centrality in the context of the theory of virtual and real is more of a co-presence, and complexity and of emergent phenomena. On the different of the possible, where potency is other hand, we can observe in details how something future to real, but contained in it, and systemic thinkers have applied network models to virtual is co-present with real, but different from it. all systemic levels, contemplating organisms as For such, we must understand that the potentiality networks of cells, organs and organic systems, in of a thing resides in the fact that any of its material the same way the ecosystems are understood as properties is updated in the future, and thus the networks of individual organisms. The same vision child becomes a man, etc. But according to the of the living systems as networks gives another virtual, there are certain properties that correspond perspective to the hierarchies of nature with its

4 distributed structure, in which life itself becomes a References network of networks. However, we could also speak of a law of development of networks based BARABÁSI, A.L. (2002). Linked: the New Science of on the Darwinian theory of networks, where the Networks. Cambridge: Perseus Publishing. stronger nodes of the network, based on the GOLDSTEIN, J. (1999). “Emergence as a Construct: context of their functional properties, will expand History and Issues”. Emergence. Vol. 11, 1999. and will be the largest and more central, at the expense of the other nodes9. And this analysis of GOLDSTEIN, J. (1999). “Emergence as a Construct: the dynamics and typology of networks allows us History and Issues”. Emergence. Vol. 11, 1999. to observe emergent phenomena in the networks JOHNSON, S. (2001). Emergence: The Connected themselves, as in the case of the ant behavior as a Lives of Ants, Brains, Cities and Software. New York: model of spontaneous self-organization in nature.10 Charles Scribner’s Sons. However, this is not totally correct, because although the machinery of cybernetic control is LISSACK M. R. (1999). “Complexity: The Science, distributed, the behavior of ants is guided by its Vocabulary, and its Relation to Organizations”. instructions, and not by laws. Emergence. Vol 11: 1999. PROTEVI, J. (2006). “Deleuze, Guattari and 4 Conclusion Emergence”. En: Paragraph. 29:2. On approaching the history of interrelationships PROTEVI, J.; BONTA, M. (2004). Deleuze and among art, science and technology, from the Geophilosophy: A Guide and Glossary. Edinburgo: materialistic perspective, that redistributes the Edinburg University Press. relationships between matter and form, and from RUSSELL, B. (1927). The Analysis of Matter. the observation of the properties of self- London: Allen & Unwin. organization inherent to matter itself, we may expose the way as art, mathematics and physics VARELA, F.; THOMPSON, E.; ROSCH, E. (1991). have interrelated, based on different space The Embodied Mind. Cambridge: MIT Press. conceptions; the way how art and WALDROP, M.M. (1992). Complexity: The Emerging telecommunications have explored the meaning of Science at the Edge of Order and Chaos. New York: networks and of remote communication; the way Touchstone Simon & Schuster how digitality and the ideal of computational mind structures the relationship between art and computer science, the thinking on the virtual in artistic practices with virtual-reality technologies, the approaches between art and software based on exploring their calculability potential and, therefore, of algorithmic programming, the theories of cognition inscribed in the relationship between artificial intelligence and artistic practices, the simulation of the emergence present in the artificial life and artistic appropriation technologies, the concept of body in the relationship between robotics and art, and, finally, the conceptions of life present in the current relationship between artistic expressions and biological knowledge, and even more especially in the development of current biotechnologies.

9BARABÁSI, A.L. (2002). Linked: the New Science of Networks. Cambridge: Perseus Publishing.

10JOHNSON, S. (2001). Emergence: The Connected Lives of Ants, Brains, Cities and Software. New York: Charles Scribner’s Sons.

Marcos SALAZAR DELFINO Laboratorio de Medios Mixtos, Centro de Estudios y Creación Artística “Armando Reverón” UNEARTE Avda. Los Jabillos, Caño Amarillo. Caracas Venezuela [email protected]

Abstract. - 1 Brief Reflection on the Anonymity This paper ponders on the identity and its technological manipulation, through The fingerprint as a pedagogic resource, the in the understanding that the preparation use of the fingerprint. of any instrument conceived for teaching is a creative fact, essentially Same, is voiced with the help of a proposes software (Photosong 1.04fm) that a way that draws itself in the passage of converts a question: in sound the gray values corresponding to the green matrix taken from a What shall be the meaning of all fingerprint photography, performed at 8 making or doing product, that links us bits color depth. to the notion of ourselves as feeling? This manipulation featured as a result in the work of artist Marcos The answer is in the didactics and is Salazar Delfino, contributes as an none other than the practice of the argument to postulate the need to autognosis or, in the same sense, "of strengthen art". attention practices in the academic processes of teaching art, with the The fingerprint is one of the recourses purpose of that identifies us as "unique in the contributing to an atmosphere of peace, world". of the anonymity’s dissolution and to the As original creation. making of stronger active participation. As deployment of constructive rhythms Keywords. - that are modeled in the mother’s womb in order to accommodate a new code of Fingerprint - anonymity - attention - art life that expresses itself and is kept - peace. registered in time as an imprint, as witness.

Thus, a picture of this fingerprint is translated into a grid of grays that becomes a

7 score of gray values that are converted I make your view mine into sounds. Mine your desire The identity is translated, is transferred, is transduced, is recorded, is I reconceive you wish transformed and it turns into unique and alike sound. And I seize your attention

Foreign but loaded of soul. Your anger

It is an inter-generative experience that Your memory only serves for the very moment, to valorize Only a host of voices aloof from any itself in its own scarcity, in its proper participation resound for you, as a result sense that transforms a pause in of remembrance and your graphic contribution, of your question. Just as the moment of art. fingerprint, of your absence.

A simple matrix of numbers between 0 It is an empty time and 255, the result of a reading of a picture Where you're not longer there taken at an 8 bits depth, translates the fingerprint into a repertoire of values to That's the eternal beholder’s agony be rhythmic and nonrhythmical interpreted The outrageous and delusional apathy as a sound experience. that suspends the courage and promotes the anonymity in front of the 90 seconds of recurrent coldness unfold technological fact as from a preconceived randomness basis I live behind a picture, a footprint, taken to sounds played on FM. behind the uproar that suspends the wisdom in the In the certainty that promotes the trade that is prompted from the aesthetic ignorance in front of the technological experience. fact; alien. Is the promotion of the anonymity that Divested of precepts installs inside the viewer; that separates her/him from her/himself. Of scruples I know a form with name and imprint. Of academic postures But it is in the fact that saves where the The apparent interactivity promoted in hope lies and resides the doorway from the inter-generative art, is nothing but the maze of impressions. an alienating power game that we authors make over the users who come In the shock itself of that that is shaken. in the search for new questions, and are seduced and The arts education in schools and trapped. universities should be

8 permanently aimed at drawing the on the possible transdisciplinary attention on this matter. dynamics in art leading us to the encounter of a synergy that, just as To raise attention towards the fact itself Mayz puts it: of what thrills beyond the mere trade that is established as from "... since it is not simply optical- the permanent flow of impressions. luminous, it not only overcomes the mind’s gambling nature... but at the Practicing the autognosis conceived as a same time, that of the subject as a training for the concentration of simple spectator of the otherness. the attention upon the bodily sensations Reciprocally - instead of being a mere in front of the work of art, is an exercise spectacle looked at, contemplated or that seen promotes peace, and in its praxis it by it - the otherness becomes deed or should accompany all the academic workmanship caused by the active apparatus that has been built around the dialogical intervention of the teaching of art. synesthesic system of the soma or body thereof”. 2 The technological art

In art, the optical-luminous and In this respect and to conclude this brief spatiality have been until now the overview of the topic fundament of almost all the composition in question, it is essential to specify in a issue. preliminarily manner the slopes However, in recent years, largely thanks lining the notion of meta-technical to the development of logos, by incorporating an extract technology, the very notion of spatiality of its creator’s work, Dr. Ernesto Mayz and distance, as well as the use and Vallenilla (1990), which shall help us conceptualization of space in art, begins keeping in mind the concept of such to transcend that traditionally visual, logos: to other notions supported in audio and tactile expressions, not only through “But within such meta-technical project the use of technical tools, but by the (which theoretic foundations instituting of new interaction and so far have not been expressively stated participation dynamics. with due rigor) can be glimpsed at least three aspects In addition thereto, the use of media in or directions of a new mode contemporary art allows us to of technical operation in front of the establish a systemic perspective in otherness... which the language of art is no longer exclusively confined to the relationship 1. - There is a first aspect which of a thematic axis and the significant expression is represented by a values ranked in its environment for set of instruments and devices that, as building up a speech, but that the manmade artificial senses, technological means used in the alter and transmute the inborn human processing of audio-visual information sensory boundaries and functions are profiled as significant themselves, - and therefore, the man's original acquiring in some cases an absolutely psychic-somatic constitution - central value in the discourse’s built up. introducing radical changes in the In this regard, it is necessary to reflect apprehension,

9 organization and intelligibilization of energy (and/or matter) trans-forming the otherness in general. and trans-muting the entities’ profile and the universe as a whole. Its ultimate By a similar procedure - as illustrated plan is to progressively convert by the use of otherness in general in a system or radars, detectors and ultrasound trans-finite energetic holos in a viewfinders or thermal sensors that constant trans-mutating becoming. guide the << vision >> of certain missiles - Such apparently alchemical purpose - the innate human body sight limits and stripped of any irrational or magical functions are modified and extended reminiscence - is the one chairing the through the use (with optical purposes) powerful and very complex technical of sonic waves or thermo-tactile stimuli. processes of the disintegration, But, just as those mentioned above, it is conversion and utilization of atomic or conceivable that for similar purposes sub-atomic energy..., as a threshold or other means or procedures can be a prelude of that outlined guiding idea used... widening in such a manner the feeding this variant. << vision >> spectrum, up to Each of these mentioned aspects - like, dimensions and substrates that result even more, their combination and absolutely inaccessible for the confluence - suppose an attitude where possibilities of the human eye. the progressive fading and parallel overcoming of the anthropomorphic, 2. - A second meta-technical aspect is anthropocentric and geocentric limits found represented by instruments or and characters of the traditional apparatuses that - upon introducing technique is demonstrated. This all changes or modifications to the layout, doesn’t come harmless or innocent. As grades we shall see in due course, their and codes of man’s (or of other beings’) projections are critical to understand own somatic and psychic structures - the radical changes taking place at the alter their congenital or innate ontology and epistemology level - and functioning…, creating or encouraging therefore, variations or innovations so much in its on the human instituting in general - on behavior, as well as in the deployment a par with our own time.” and distribution of the energies thereof.

The meta-technical instruments, in this It is in that progressive fading of the sense, build a new otherness... boundaries and anthropomorphic, which own ontic-ontological structure anthropocentric and geocentric imposes concomitant variations so characters of the traditional technique, much where the cause reinforcing the in its eventual epistemological anonymity is founded. But we no longer objectification and intelligibilization, as talk of the anonymity in front of the well as in the field of its built and otherness, of concealment of the face, redesigned teleonomy. of name, of whichever any identity form. 3. - The third and final aspect - perhaps the most complex and recent of them all We speak of anonymity in front of the - mind‘s mirroring experience, as is constituted by instruments or devices opposed to the ‘myself’. that limitlessly transubstantiate the

10 It's like when you lose the time of the and promotes the anonymity. stroke, the craftsman’s accomplished doing, 4 Acknowledgments the violinist's instrument strike; this is when the aurous being’s fingerprint My thanks to: vanishes, is absent and the Colegues from Universidad Nacional metanarrative of the authorship and the Experimental de las Artes. identity loses its symbolic effectiveness. Departamento de Diseño, Arquitectura y Artes Plásticas de la Universidad Simón Bolívar. 3 Conclusions Sra. Isabella Delfino Dra. Blanca Travieso 90 seconds of recurrent coldness Prem Pal Sing Rawat (por revelar el sentido) in the certainty that ignorance promotes in front of the technological fact References Ernesto Mayz Vallenilla, Ratio aloof therefrom Technica. Caracas: Monte Ávila Editores, 1983. divested of precepts Ernesto Mayz Vallenilla, Fundamentos of scruples de la Meta-Técnica. Caracas: Monte Ávila Editores, 1990. of academic postures Ernesto Mayz Vallenilla, El problema just a pile of voices strange to all de América. Caracas: Universidad involvement Central de Venezuela, 1959 delighted in its absence full of sounds José Ramón Ortega y Gasset, Meditación de la Técnica . Madrid, in an empty time where the passing Ediciones de la Revista de Occidente, thereof only distances from poetry 1968.

Juán David García Bacca, Elogio de la from the dying axiom ethics has Técnica. Caracas, Monte Ávila condemned Editores, s.f.

to agonize forever Martín Heidegger, La Pregunta por la

Técnica. Idec Digital, outrageously http://idecdigital.arq.ucv.ve:8080/tym/ in the apathy’s clash that suspends Alfredo Vallota, Consideraciones wisdom acerca de la percepción sensorial en la Meta-Técnica. Idec Digital, that suspends courage http://idecdigital.arq.ucv.ve:8080/tym/

Future Potentials for ASCII art CAC.3, Paris, France

Anders Carlsson A. Bill Miller ChipFlip Penn State Altoona Box 1541 3000 Ivyside Park Bräkne-Hoby, Sweden, 37010 Altoona, PA, USA 16601 [email protected] [email protected]

Abstract Reynolds (2011) calls retromania, the large-scale ASCII art is a text-based expression that cultural development that re-approaches previously traditionally is concerned with remediating popular forms of expressions in order to explore images, words and objects. The paper “new” options. describes its historical connections to poetry, programming, literature and hacking and 2 Brief History defines ASCII art as a genre, consisting of several categories. The authors identify a Before typewriters and computers, calligrams were number of movements towards novel forms of likely the most common form of text art. A calligram ASCII art that explore medium specific is an image built with words or text characters that characteristics for drawing, design, advertising relate to the content. Simmias of Rhodes’ Axe (325 and conceptual art. The authors also argue that B.C) is one early example. Micrography is a Jewish the popularity of digital media, which for form of calligraphy, which is said to be a method to example leads to scarcity in URLs, increases bypass the restrictions on images of the Second the potential for ASCII art to play a more commandment. In some examples of printed texts, illustrations were also created through the careful active role in human communication. organization of typographic ornaments1, note at end of page. Keywords ASCII art, textmode, graphics, ANSI The typewriter disseminated text art to a broader public. A secretary by the name of Flora Stacey made 1 Introduction a famous picture of a butterfly in the 1890s, by moving the paper in the typewriter while typing. In his book Gramophone, Film, Typewriter People such as Julian Nelson and Alan Riddell Friedrich Kittler identifies the typewriter as the popularized this technique for hobbyists as well as symbolic medium par excellence (Kittler 1986:15). artists and poets. Sylvester Houédard was a priest and With a highly limited set of characters, humans concrete poet who used typewriters to make abstract have managed to formulate grand texts to art.2 Like Contemplation (1972) shows many abstract symbolize laws, religions, politics and myths. But shapes, which gives little indication of being made the material characteristics are normally invisible with a typewriter. to us. This is why the typewriter can be considered In 1921 Marcel Duchamp and Man Ray produced as a key symbolic medium. It paved the way for the magazine New York Dada, well known for its the computer keyboard which at the moment, using cover which displayed a small readymade collage the Unicode standard, supports up to 110,000 surrounded entirely by the typewriter generated characters. words New York Dada backwards. In this example Text characters have been used for visual the words become a pattern and incantation that art for thousands of years, and seems to be gaining reinforces dada aesthetics (Ades 2006:146,158). Theo momentum. One reason is that digital media, by Van Doesburg, more likely known for his paintings virtue of programming languages and protocols, is and contribution to De Stijl, created the visual design tightly connected to the text characters of e.g. for the four-issue dada-constructivist magazine ASCII and Unicode. Studying the foundations of 1Examples are available at digital media, often indirectly means to study flows http://pplspcoll.wordpress.com/2011/05/02/analog-ascii/ of text characters - just like in the Matrix, 2 See for example Fun With Your Typewriter by Marge according to some. Another reason is what Roemer, 1956

13 Mecano. Each issue included the name of the important aesthetic precedent can be found in the magazine as a text-diagram and presented a direct rationalization and atomization of visual relationship between the the type characters, the communication in the 1920s - both among artists image design they created, in addition to being and marketers (Manovich 1993:11). Later on, conceptually significant (Ades 2006:261,270). Dada pattern artists such as Peter Struycken or artists’ interest in the use of language perhaps created Dominique Douat would generate early computer a unique avenue for experimentation with printed text graphics with similar ideas, also connecting to the that was visual, verbal, conceptual, and sometimes 18th century works of Sebastian Truchet.7 political. It is also worth mentioning tile-based graphics, Concrete poetry functions by placing equivalent common in videogames in the 1980s. In order to importance on the form that the typographical save resources, graphics were constructed from a characters are given and their semantic meanings. An set of tiles, similar to a font but often using more early and popular example of this is Guillaume Apollinaire’s Il Pleut (1913-1916) where the meaning colours. People like NoteNdo who hack old game consoles therefore often end up with a mosaic of the text is enhanced by its diagonal and linear 8 presentation. This approach was also employed by aesthetics similar to textmode. The similarity many artist/poets, such as bpNichol3 . Other Literary comes from the consistent grid that is used to movements, including Oulipo, experimented with either display graphic tiles or fixed-width type language and text using algorithmic methods like characters. Raymond Queneau’s Cent Mille Milliards de Poèmes (1961) in which the fourteen lines of ten sonnets are interchangeable4. The configuration of the printed text is based on how the reader interacts with it. These examples demonstrate the interest of artists and poets in how we interpret the printed word based on its presentation and design. There is the potential for meanings to be generated through the visual organization of the text and the language used that can reinforce each other as well as become conceptual investigations. In his historical analysis of pre-1900s visual texts, Pattern Poetry, Dick Higgins refers to this relationship as the “synthesis of visual and literary experience” (Higgins 1987:3). Sebastian Truchet, 18th Century 2.1 From Teletype to ASCII: Textmode Teletype, which appeared around 1850, was the While typewriter art allows the user to position text first telematic medium for text communication9. It characters arbitrarily by moving the paper, ASCII was similar to morse telegraphy in converting text art traditionally works in so called textmode. A characters into sounds, but it used typewriters and textmode screen is a grid of equally sized cells, printers as interfaces. The oldest preserved teletype where each cell can contain one text character. It graphics are from the 1960s, but the practice is most uses a monospace typeface, meaning that all the likely much older than that. In order to standardize characters receive exactly the same amount of the communication, teletype focused on the Baudot space, allowing for the same amount of characters- code which predated the ASCII-standard by almost per-inch or line.5 The number of characters per line 100 years. usually varies between 32 and 80. Modern text inputs use a variable-width typeface, which is not ASCII appeared in the 1960s - the same decade as possible in textmode.6 As such, textmode graphics the first computer art exhibitions. There were computer artists/engineers who worked with similar can be described as a form of mosaic that uses a methods to ASCII art. Ken Knowlton and Leon strict grid and a set of typographic symbols. An Harmon exhibited a textmode version of a nude at the 3 Numerous examples exist on the artist/poet’s website : 7 http://www.bpnichol.ca/ For more on grid based art see http://radicalart.info/AlgorithmicArt/grid/index.html 4 Interactive online version available at 8 Examples of noteNdo’s work can be seen on the artist’s http://www.bevrowe.info/Queneau/QueneauRandom_v4.html website http://notendo.com 5 More on non-grid based ASCII art and proportional fonts see 9 Royal Earl House and David Edward Hughes introduced http://www.andrewhurle.com/17/17_pages/17.html keyboard-based telecommunications 1844-1857. 6

14 MOMA in New York in 1968. By calculating the restricted conditions, such as making a “music video” brightness of each text character using a computer, using only 1 kilobyte, and would therefore often use they converted a photograph into a mosaic-like textmode graphics together with chipmusic. Also textmode picture. It is the same technique used by e.g known as chiptune, this genre uses digital waveforms ASCIImeo.com and the VLC media player, to found in the soundchips of old computers and convert video into ASCII animations. consoles, much like text graphics uses the fonts of these systems. Since there are both technical and 2.2 Replicating the Pixel cultural connections between the two, chipmusic and ASCII art frequently appear together in demos, At the end of the 1970s, text art was popularized crackintros and keygens (programs that generate through two digital media: computer games and software licenses). teletext. Early computers like PET, TRS80 and ZX81 were usually better at using text rather than pixels, On the Internet, ASCII art most likely reached its which made them suitable for text adventures high point of importance before it was available to (interactive fiction). The character sets of the the general public in the early 1990s. At that time it computers had symbols that were well-suited for was distributed mostly through e-mail listservs and graphics. For example, several platforms added BBSs (Danet 2001:208). Connection speeds were symbols that enabled a technique similar to pixel slow by today’s standards and transferring the smaller graphics. Larger custom texts and images could be size text files was done much more quickly than a generated by combining symbols that represented traditional image file. ASCII art was a primary means different portions of text characters. One ‘pixel’ was to ‘decorate’ or display images content. Later, a new one section of a text character. form of ASCII appeared which doubled the amount of available characters. These new characters, along Teletext worked in a similar way. It was a with ANSI escape codes that enabled the use of TV-service with text-based information sent in colours, created a new generation of ASCII art that is parallel with the standard TV-signal, originally usually referred to as ANSI art (see Hargadon 2011). intended for subtitles. The graphics were completely A number of Usenet newsgroups developed and text-based, built with a graphics-friendly character set further distributed ASCII art. which was also used in home computers such as ABC80 and MicroBee. The teletext standard was also The World Wide Web allowed for access to ASCII used for videotex, which was a modem-based art in a different way. Originally, those involved in precursor to Internet that allowed for two-way newsgroups and listservs made their work available communication. In France, this service was in use to others in their groups through FTP sites. Files had until 2012.10 to be transferred and then viewed offline. The web enabled artists to show text graphics in online 2.3 Networked Communications “galleries”. The work could be seen on the site and copied or downloaded as needed. In addition, it Around the same time, the home computer revolution became easier for artists to share technical brought with it a modem-connected network of information about their process. ASCII art images Bulletin Board Systems (BBS). Along with e.g and FAQ files were combined into large collections videotex, they were the social networking sites of the and website databases allowing for the accessibility 1980s and 1990s where users exchanged messages that has made them now ubiquitous. and software. The interfaces were completely text- based, which created a demand for high-quality In the art world, text-based network media were ASCII menus and logos. In the early 1990s, these normally used more for words than images. graphics started to appear as separate artefacts, Exhibitions such as Interplay (1979) and La Plissure known as ascii collections or art packs (Polgár Du Text (1983) experimented with networked 2005:121). It used a rather distinct mixture of graffiti, storytelling in a similar way to some mail art, but in leetspeak poetry and pixel art aesthetics. This ASCII real-time. In 1997 Steve Deitz began directing art style is still mostly seen in hacker contexts on Gallery 9, one of the first specifically digital FTP-sites, IRC and in read_me files for cracked collections housed within a more traditional software.11 contemporary art museum, the Walker Art Center and included works like Mark Amerika’s PHON:E:ME.12 ASCII art became very popular in the demoscene, Hypertext and experimental electronic text more which is a subculture dedicated to audiovisual commonly fall within the digital humanities, since hacking that at the time depended on BBSs for quick they are generally more concerned with semantic distribution. They frequently worked under very meanings than visual appearance.13 It is also worth 10 Examples of artistic and commercial works can be found at noting the practice of generating larger letters and text-mode.tumblr.com 11 Textfiles.com is an extensive resource on this topic. 12 http://phoneme.walkerart.org

15 fonts, such as the FIGlet collection14, out of smaller necessarily interpreted, it is processed. All text and ones. images that are displayed are converted through standards like ASCII into a format that is more easily One of the more widely known ASCII artists is understood and processed by the computer. The use Joan Stark whose collection carries a wide range of of ASCII in early Internet Art was a way of revealing works, most of which include her signature ‘jgs’. Her what is going on within the machine and perhaps a site is still hosted by one of the earliest free web way of making it aesthetic. Vuk Cosic’s Deep ASCII17 hosting services Geocities.15 It is a good example of a (1998) demonstrates this by exploring the common format for sites focused on ASCII art in that relationships between the porno and game industries it includes a collection of ASCII images as well as by developing a method to display video images with information about the history of the form, FAQ, ASCII on a Pong game console. The graphic tutorials, and links to other ASCII collections. Stark’s simplicity of green text on a black screen, although work is largely done ‘by hand’, meaning that the unnecessary in the late 1990s, reinforced the aesthetic artist created the images by typing in text editing quality and history of writing code - a language software. This relates directly the tradition of meant to be interpreted by a machine but instead typewriter art within a textmode grid while other displaying one of the most recognizable pornographic opportunities exist that use software to convert films as a sort of visual raw data (Figure 3). The site images into their text equivalent. Because of Stark’s ASCII Rock (2003) by C505 (Yoshi Sodeoka) method, the images have an elegant and engaging combines ASCII art video processing of popular rock simplicity of form. Browsing her site reveals the music videos accompanied by General MIDI music.18 reason she is commonly referred to as the ‘Queen of In these examples, ASCII art moves away ASCII art’(Figure 1). from the line-based aesthetics of e.g. Joan Stark in order to explore the conceptual relationship between 2.4 Internet Art uses of ASCII text-as-code and text-as-image. Stark’s ASCII is a Loading the website of the artist duo Jodi (Joan form of manual craft made by hand, where the Heemskerk and Dirk Paesmans)16 creates something success of the work lies in a correct remediation, or that reads like a glitch - it appears as gibberish, as if stylish interpretation. With Jodi, Cosic, and Sodeoka the site creators didn’t debug the page. It is an that translation is automatized and less significant. underscored, center-aligned block of neon green text, What matters is what has been translated, and why. unreadable and incomprehensible (Figure 2). A casual Both types of work maintain references to the subject observer today might still read the page as if it was and are therefore still representative. However, with simply a mistake. It is as if the page that has loaded is the more conceptual ASCII work, our attention is nothing more than random configuration of text drawn towards how computational processing relates characters. Behind this assumed mistake, in the to human language and visual perception. source code, we find something much more meaningful. The potential that is explored with this work is held between the code and display levels of 3 Terminology and Categories the browser. It exploits the fact the code level ASCII, Morse code and Unicode are character whitespace is ignored when the web browser renders encoding systems (also known as character sets) that the page. Jodi’s source code shows us ASCII art assigns numbers to every symbol.19 The symbols are diagrams. The page is a bomb and we see the letters, numbers and cultural glyphs like arrows and aftermath within the browser window and the bomb squares, but there are also non-visible control itself in the source code. The ASCII art is hidden characters such as tab, return and clear screen. These away from view and we see the text characters control characters date back to Baudot and Morse unformatted where we typically expect to see codes.20 The characters can look different depending something legible (Greene, p. 41). on which typeface is used and what size it has, and The approach to ASCII employed by Jodi is the characters can be positioned in a textmode grid or typical of the way that artists working on the early more arbitrarily. The appearance of ASCII art can Internet adopted and used existing media for creative also change according to the screen used. For and conceptual experimentation. At the core of their example, an old cathode-ray screen can be important approach was the realization that within the machine, for the experience of the graphics, as Ian Bogost a computer, the meaning held in language is not 17 http://www1.zkm.de/~wvdc/ascii/java/ 13 See Electronic Literature Organization at 18http://www.c505.com/vvvvvv/projects/ascii_rock/index.html http://eliterature.org 19 A list of Internet-compatible character sets can be found at 14 http://www.figlet.org/ and http://is.gd/UIDCKe http://www.iana.org/assignments/character-sets 15 http://www.geocities.com/spunk1111 20 For example, BEL is a code that usually makes a warning 16 http://wwwwwwwww.jodi.org/ sound which originates in the Baudot code.

16 argued with the television simulator.21 From this it is an 8-bit character set which contained several possible to define several different levels to discuss characters that enabled a pixel-based approach to ASCII art. ASCII art. Block ASCII is essentially a black and white form of ANSI art, and it was very popular Symbols → Character Encoding → Typeface → among MS DOS users, particularly in the “warez Typography → Screen → Perception scene” (see Rehn 2001). ANSI art uses 8-bit encoding23 and offers 16 ASCII originally referred to a standard from the predefined colours. It is arguably the ASCII art that 1960s which encoded 95 printable characters, but the is most similar to pixel graphics. The combination of meaning of the term has shifted over time.22 Today, Block ASCII characters and colours creates an the term ASCII normally refers to any text-based appearance similar to pixel graphics and some expression, regardless of the character encoding characters that mimic dithering. The underground system used. For example, text-based art online is community of artists that work with ANSI is often called ASCII although it most likely uses Unicode. called The Art Scene, and they release art packs24. The term ASCII art is used accordingly in this text; as With each of these styles of ASCII art, it is an umbrella term for any text-based art that uses a easy to see the relationship between the text preset font in textmode. As such, we consider ASCII characters used and the way in which pixels were art as a genre which consists of several categories. eventually used for graphical display. The different Some of those categories are direct consequences of styles reproduce the concept of pixels in different the platforms used, such as PETSCII (early ways because of the different character sets they Commodore computers) and ATASCII (8-bit Atari utilize. For example, something created in a Block computers). These terms refer to both the character ASCII style might use something like a ‘dark shaded encoding and typeface. Shift_JIS is a similar term, block character’ ▓ and is visually similar to the used in the west to describe Japanese text art (in PETSCII character ▒. Japan usually called AA, short for Ascii Art). Technically, Shift_JIS is a group of character 4 Non-representative ASCII encoding systems for Japanese characters. They are It is a natural tendency for images to be recognizable used with various typefaces - both monospace and and interpreted easily. For ASCII art this means that variable - and are not fully compatible, which has the visual experience outweighs the verbal made Unicode encoding more popular in recent connotations of text. In some cases the text used is years. significant because it reinforces the purpose or Amiga ASCII uses a 7-bit character encoding and meaning of the image created as in concrete poetry. has a particular style due to fonts like Topaz and P0T- ASCII art is normally successful when it renders N0oDLE which made it possible to make continuous correctly, it is decoded properly, the communication lines with /-signs. Therefore, skewed logotypes are is received successfully - as with most media art. But more popular here than in other ASCII art. In the ASCII art can move beyond its traditional functions early 1990s, dedicated ASCII art groups started to and expand while still carrying its aesthetic release collections of their work, which can be significance. viewed online with the right encoding and fonts at Text-based visual art is commonly not meant asciiarena.com. to be executed or interpreted as in software and language. It could become simply a visual language on a screen made up of both code and image marks to be interpreted by the viewer. This works on the basis of imagination and recognition: that the code is not code in the typical sense but is preformatted to generate an image or graphic. It works in an avenue between writing and drawing; not explicitly one or Razor 1911 by Karma. Taken from asciiarena.com. the other but with aspects of both. It is decoded by the viewer as with the tradition of other image forms Block ASCII is the informal name given to a like painting or drawing. The marks are decoded and standard developed by IBM in the late 1970s. It was an image constructed individually by the viewer. The 21 http://www.bogost.com/games/a_television_simulator.shtml potential for non-representational ASCII art begins to 22 ASCII contains a total of 128 characters, but 33 of them are control characters which are used to affect how text and space 23 is processed. The original meaning of the term ASCII is today http://en.wikipedia.org/wiki/Code_page_437 known as US-ASCII. 24 http://16colours.net

17 form around the consideration of the textmode character constraint of Twitter (Figure 4). The posts characters as marks on a digital surface. under this tag include type graphics, cartoons, The ASCII character set is a limitation drawings, and simple patterns. The works crashtxt unique to digital environments and in contemporary and reCAPCHAT by jimpunk use Twitter to different uses of computers it is somewhat arbitrary. In most conceptual ends.26 The former presents the viewer cases, it is irrelevant to know or understand the with a web-based Unicode keyboard and processes character set because it happens ‘behind the scenes’. the input directly into the crashtxt Twitter (Figure 5). However, this characteristic is what ties together The latter asks users to use a traditional ‘captcha’ experimental literature forms like concrete poetry and form and posts to the reCAPCHAT Twitter.27 These Oulipo with the conceptual aesthetics of early works share the modularity of ASCII art but not its Internet artists. The constraints placed on visual traditional, pre-Unicode aesthetics. Our attention is display by a deliberately chosen set of text characters turned to the way we see and interpret textmode opens the potential for a digital aesthetic that speaks characters on the web. about how humans use language different than Other artists use the characteristics of social machines. It is perhaps a visual language somewhere networking sites as an invitation to look for and between them that creates avenues for new meanings exploit bugs with aesthetic intent. The Facebook to be generated. With this understanding, the ASCII account glitchr run by Laimonas Zakas, has been character set becomes like a digital mark-making tool known to break outside of the normal formatting and that could be used as a more complete analog to layout of the networking site.28 Posting status updates drawing or painting than software remediations of made up entirely of obscure Unicode characters and brushes or pencils. As the typewriter and letterpress diacritical marks, glitchr’s feed was made up of opened up room for experimentation with text as floating marks over, around, behind, and on top of image, so does the keyboard. other page content (Figure 6). The posts, in essence, One of the major characteristics of new broke or hacked the Facebook layout and transformed media identified by Lev Manovich is that of it into a place where text characters were “emptied modularity (Manovich 2001:30). In the context of from their primary communicative signifiers and ASCII as a standard, the individual text characters are repurposed as formalized, aestheticized objects”.29 modular and portable between almost any software Screenshot documentation of the Facebook account application. In addition, entire ASCII art and Tumblr eventually served as a way to preserve compositions are modular and can be copy/pasted the posts as the websites fixed the bugs glitchr intact from one document to another or from exploited.30 What became visually challenging and application to application without changing the aesthetic was the breakdown of the tightly designed fundamental text nature of the composition. For site called forth through the insertion of a form of example, one could copy a drawing directly from ASCII art where normally one might find banal text. Joan Stark’s site, paste it into a text document and Daniel Temkin’s work Unicode Frenzy edit, then copy the edited version into animation randomly chooses Unicode characters and organizes software and create a moving typographic sequence. them into multi-layered designs.31 Like glitchr’s At each stage of that process the drawing is a broken Facebook displays, Temkin’s Unicode layouts modular unit that functions across software turn our attention to the aesthetics of individual text applications or operating systems. It is a modular characters and how they are arranged in web display structure within a textmode grid. (Figure 7). Disregarding the type grid and the preformatted nature of traditional ASCII art, their

4.1 Current Tendencies 26 Available at http://jimpunk.com/reCAPCHAT/ and ASCII art websites still exist and their communities http://jimpunk.com/crashtxt/ 27 thrive.25 In addition, new uses of the format have https://twitter.com/#!/crashtxt, https://twitter.com/#!/reCAPCHAT been taking form on other Internet-based platforms. 28 https://www.facebook.com/glitchr Under the hashtags #TWITTERART, #140art, and 29 http://rhizome.org/editorial/2012/mar/29/diacritics-glitchr- #sYmBoLaRt, a range of users draw within the 140 draft/ 25 Some examples include utf8art.com, chris.com/ascii, 30 http://glitchr.tumblr.com/ alt.ascii-art, asciimator.net 31 http://danieltemkin.com/UnicodeFrenzy/

18 work generates a visual aesthetic from broken popular. From that perspective, ASCII art will likely language. A more simplified way of viewing this gain momentum over the next few years with breakdown might come from another project of potentials for many different artistic, practical, Temkin’s, Unichar, which creates a sort of ‘Hot or political and corporate uses. Not’ for the Unicode character set.32

5 Conclusion References The relevance of ASCII art falls within its unique [1] Ades, Dawn (2006) The Dada Reader, A position in between human language, programming Critical Anthology. University of Chicago Press. language or code, and visually aesthetic language or image. Its potential, however, lies in our ability to use ASCII as a way to blur those boundaries. When text [2] Danet, Brenda (2001) Cyberpl@y. Berg. characters are used independently from their function as language and when images are generated with no [3] Hargadon, Michael A. (2011) Like City Lights, representational reference, then text characters Receding: ANSi Artwork and the Digital become their own modular visual units. Those units Underground, 1985-2000. Concordia University. might maintain their other significances, but can also generate new meanings. [4] Higgins, Dick (1987) Pattern Poetry: Guide to These potentials are already explored by artists, but an Unknown Literature. State University of New will likely spread to other fields. With the popularity York Press, Albany. and power of text-based media such as Google Search, Twitter and Facebook Ads comes new possibilities and requirements for strategic [5] Kittler, Friedrich (1986/1999) Gramophone, communication. Companies, organizations and other Film, Typewriter. Stanford University Press. brands can improve their visibility through ASCII art. One example is the car rental company Sixt who [6] Manovich, Lev (1993) The Engineering of added ASCII art to Google AdWords in order to Vision from Constructivism to Computers. receive more attention (Figure 8). Following this University of Rochester. logic, it is not surprising to see artists and fashion designers incorporating ASCII art as print on clothing.33 [7] Manovich, Lev (2001) The Language of New Media. MIT Press. Similarly, the scarcity of available URLs and account names have led to difficulties in registering new unique names. ASCII art is a way to tackle this [8] Polgár, Tamás (2005) Freax - The Brief History issue, which so far has been most notably in music. of the Computer Demoscene. CSW Verlag. Prince replaced his name with a symbol in 1993, and more recent examples include acts such as :), ??? [9] Rehn, Alf (2001) Electronic Potlatch - a study and !!!. In so called witch house it is common to use on new technologies and primitive economic names with odd Unicode characters, like †‡† and behaviors. Royal Institute of Technology. ▼□■□■□■. According to Matt Sheret at Last.fm, “it’s not so much anti-Internet as very clever Internet” since it makes it easier for the fans to find [10] Reynolds, Simon (2011) Retromania - Pop the music (Watercutter 2011). But in order to talk Culture’s Addiction to Its Own Past. Faber & Faber. about these names, there is a need to develop words or sounds for it. [11] Watercutter, Angela (2011) Crazy Characters In that sense, ASCII has gone from object to Help Indie Bands Outsmart Google, in Wired subject. It is no longer only an invisible transmitter, Magazine, January. but an active ingredient in the way humans http://www.wired.com/magazine/2011/01/pl_music communicate. Not only is our online language and _ungoogle behaviour conditioned by Unicode, but our verbal language is affected by which Unicode characters are

32 http://danieltemkin.com/Unichar, http://hotornot.com 33 See Kayla Mattes ASCII Catz : http://www.kaylamattes.com/index.php?/knit/90s-knit- collection/

19 6 Ilustrations

Figure 1. Joan Stark

Figure 2. Jodi

20 Figure 3. Vuk Cosic

Figure 4. #TwitterArt

21 Figure 5. jimpunk

Figure 6. glitchr

22 Figure 7. Daniel Temkin

23 Figure 8. Sixt ASCII adwords

24 Hyper-Production and the Value of Exquisite Corps on the Web

Blanka Earhart The Planetary Collegium, Plymouth University 912 E. 3rd St. #404 Los Angeles, CA 90013 USA [email protected]

Abstract principal, which creates an aesthetic specific to the digital world. The real and the virtual worlds around us are The genesis of the project comes from a becoming louder while generating change at an reflection on the intensity of the world unprecedented rate. The great volume and surrounding us. The sheer volume and pervasiveness of information, products and pervasiveness of information repeated experiences distributed throughout multiple throughout many platforms creates a sense of media creates a need for faster response times pressure for an individual. The sensory stimuli and greater production volume. In this available to us either through entertainment, atmosphere, an individual finds him/herself transportation or by simply living in the world under pressures to deliver in a similar manner. provides intensity previously unmatched by life. In order to perform in the heightened economy, Our environment has grown louder and more the production, taking place on the Internet, is aggressive in demanding our attention. In an expected to grow while its value drops. effort to meet the challenge coming from the Throughout the history of the Western world we outside, we amplify and distribute ourselves as have seen different permutations of the well. The Hyper-Generator is a project that phenomenon of labor. Today, we are finding facilitates self-amplification through ourselves addressing the old questions in the multiplication of the effects a single person has new electronic medium. An artistic intervention in the world. provides a potent background for a discussion of This intensification finds its reflection in a labor and play in the virtual spaces. number of human endeavors such as work, play, Keywords economic and social currency and modes of expression. In this paper, I will first examine Internet, Social Media, Production, Invisible the state of labor and production relationships in Labor, Leisure the amplified economy. Then, I will formulate the essential issues addressed by Hyper- 1. Introduction Generator as they relate to the amplified economy. I will conclude with a possible As the demand for faster and more expansive direction to approach expression in the digital production on the web increases, the labor world. performed is marginalized and often reclassified as a leisure activity. This state of affairs 2. Hyper-Labor inspires a reflection on possible outcomes of this trajectory. Often, individuals express their Throughout the history of the Western world we desire to clone themselves, figuratively have seen different theoretical approaches to the speaking, in order to meet the demands for phenomenon of labor. The term initially coined productivity. Fortunately, personal production by Marx in the context of class struggle stood can be magnified through the increase in the for commodity that can be bought and sold. result of one’s actions. The Hyper-Generator is “Labor does not only produce commodities; it an art piece creating an environment conducive produces itself and the laborer as a commodity to such production. The result is an and that to the extent to which it produces agglomeration of data arranged according to a commodities in general.” (Marx, 1977) Further, 25 Marx explains how such conceived production variety and usefulness of the content remains escapes the original worker and becomes hidden behind the screen. alienated. “What this fact expresses is merely Online work is organized along two major this: the object that labor produces, its product, trajectories: the development of the confronts it as an alien being, as a power infrastructures such as specific platforms and independent of the producer.” (Marx, 1977) production of content. Because of the The alienation of the product from the producer ephemeral nature of the medium, it’s lack of leads to a schizophrenic situation, in which the tactility and cryptic development process, the maker is not entitled to fruits of his/her efforts understanding of work required escapes most nor the work itself. In this gesture the laymen. This leads to unrealistic expectations relationship between the worker and the product and the notion that only minimal work is is distorted such that work does not appear to required in designing, building and maintaining originate from the worker. online structures. Somehow this This line of thought has been taken a step misunderstanding is perpetuated and further by Second-wave Feminist thinkers. They exaggerated to a point where the development adopted the Marxist rhetoric to talk about the process is expected to be economically invisible labor of women and minorities in order minimized, if not rendered completely invisible. to expose the social bias against work performed In parallel, the production of content is by those groups. The work done at home and at increasingly performed by unpaid labor. This menial jobs was and some would argue still is tendency reaches its peak in social networking not rewarded with pay or high social reputation sites, where personal information and content is despite the skill and dedication it requires. A handed over to the site’s proprietors by means woman cannot be even alienated form her of a contract. In effect, we are building complete domestic work because the work is not demographic profiles of ourselves for the acknowledged as real. “The more successfully advertisers turning ourselves into a commodity women perform this concrete work (…), the as a worker and as a product. This is also the more invisible does their work become to men. point where leisure becomes indistinguishable Men who are relieved of the need to maintain from work. While in keeping with the their own bodies and the local places where they traditional approach, one is not being paid for exist can now see as real only what corresponds their leisure activities, the proprietors of the to their abstracted mental world.” (Harding, content still capitalize on the products of those 1996) Thus, the domestic labor fails to be activities. The work is contextualized as play. conceptualized as an activity that generates Since there is no monetary value placed on the value. I believe that there are parallels between products of play, the process by which they the labor performed on the web and the invisible come to exist becomes economically invisible. labor as defined by Second-wave Feminism. The distinction between leisure and labor Today, the economy of labor is undergoing activates may not be a sufficiently complex another shift. The majority of the population in conceptualization when it comes to online the developed countries lives immersed to a content production. The feminist theorists have different extent in the virtual world provided by been struggling with a similar delineation the Internet. Similarly to regular life, the virtual applied to housework. Sandra Harding talks medium requires work in order to develop, about the example of time-budget studies function and thrive. Simultaneously, the virtual “which regard housework as part leisure and world subtly infuses every aspect of our lives. part labor – a conceptualization based on men’s Through our constant use and reliance on the experience of wage labor for others vs. self- medium, we often forget its complexity and are directed activity. But for wives and mothers, only snapped back to reality by an annoying housework is neither wage labor nor self- error message or dysfunctional user interface. directed activity.” (Harding, 1996) The The workforce ensuring the smooth operation, distinction coming from the outside male

26 perspective fails to represent the experience of demands are still becoming more ambitious. the person performing the activity. We are reaching the limits of human capacity in Similarly, maintenance of multiple social the final stretch for hyper-production. profiles, as experienced by an average user In response to the high demands for volume, (with the exclusion of advertising low price points and speed, one attempts to professionals), is neither leisure nor wage labor imagine the world capable of providing just that. activity. There is a certain peer pressure to keep The Hyper-Generator is an environment, which up with the production of fresh content, which translates casual actions performed by a human feels like work, yet the activity is performed in a course of going through one’s day into during the time when one is not working. This online production. The result of one opening a ambiguity allows for manipulation of what it refrigerator or turning on a water faucet is means to produce content on the Internet. The recorded on the web as a concrete action or final conceptualization is dictated by the number of actions, which amount to producing a pervasive power structure. In case of the profile on one or many of the social networking feminist view, it was provided by the masculine sites. This human enabled system arranges perspective while in today’s online economy, content without direct human intervention and this judgment is passed by the owners of the produces assemblages shaped by the rules of the digital platforms. In either case, the specific site. While the content is pulled worker/producer has no say in the economic randomly form the web, the action mimics user categorization of their own efforts. behavior, in which different pieces of content It is additionally interesting to note that the are sampled and liked becoming part of a single sheer volume of online production undermines profile. quality. In fact, quality becomes an obsolete Beyond the increase in production volume, concept no longer useful in a world of countless the Hyper-Generator also results in an increase permutations. The reshuffling, re- in intensity of each action. In such system, each contextualizing and reclaiming of the content action having multiple results becomes more already available is the new pass time or and louder than its original. The labor that production. By the virtue of exercising one’s serves as the catalyst is not only optimized but taste, namely liking something, one brings an also amplified through the increased production. object into one’s sphere and incorporates it into Additionally, actions become distributed across the personal collage. The act seems frivolous platforms and encrypted as results not but is latent with rich economic meanings, reminiscent of the original. Therefore, the which in turn become monetized. Of course, system accomplishes two tasks: increase in this activity retains its economic significance productivity while hiding the labor even further. only as long as it’s tied to a living, breathing and I imagine this perverse exaggeration as a buying human being. possible result of the trajectories, which we are already following as described above.

3. Hyper-Production 4. The Exquisite Corps

With the advent of the invisible labor on the The profile created by the Hyper-Generator web and the recent economic meltdown came unfolds in a manner similar to an exquisite additional pressures to perform continuously at corps. The random information from the web is peak capacity. The web is viewed as a pulled together into a common context. The phenomenon that comes to be almost silhouette of the profile serves as the unifying spontaneously and above all mostly for free. logic and suggests connections between the Yet in order for this perspective to hold true, arbitrary parts. Thus created profile has the legions of developers and content producers capacity to go through a multitude of work around the clock and around the globe. In permutations in a gesture that suggests the mean time, the pace is only increasing and

27 collaboration, which is not aware of it An exquisite corps as a design idea implies a happening. possibility of data aesthetic understood on its The trope of an exquisite corps embraces own as just data. The Dadaists invented their randomness and poetical serendipity. Both of game in an effort to move away form the those follow the modernist ideas of world as a tradition of mimicking the real world in art. As coherent entity, where the mind creates Andre Breton energetically postulates: “These connections even in a face of discord. On one drawings represent total negation of the level, the created profile is a pure random data ridiculous activity of imitation of physical inserted into prearranged categories or a characteristics, to which a large and most template. Then, the human viewer employing questionable part of contemporary art is still interpretation performs work, which anachronistically subservient.” (Breton, 1948) reestablishing the psychological continuity of In a similar way, data is still harnessed in the the whole on a purely instinctive level. Andre service of communicating and mimicking an Breton, one of the inventors of exquisite corps, outside world, which the data is not. In effect, talks about this suspension of reasoning mind as we can never deal with the digital world as such a success. “Finally, with the Exquisite Corpse since we experience it as a metaphor of the we had at our command an infallible way of physical world. Perhaps, rearranging data is the holding the critical intellect in abeyance, and of first childlike gesture that honestly deals with fully liberating the mind's metaphorical the medium itself. activity.” (Breton, 1948) The Dadaist, at the beginning of the last 5. Conclusion century, worked in a context of a society already filling up with information and disturbances The Avant-garde project claimed its place in the such as World War I. Their reaction to the society by eluding established constructs and increasing complexity of the world was to challenging existing modes of expression. positively make sense of it through play and These actions have been rendered futile every discovery. Our present paradigm proves that it time the established institutions and economies is possible for the information to accumulate reabsorb such gestures into the mainstream infinitely. Faced with this predicament, can we culture. The Hyper-Generator creates products, still embrace the discovery process and draw which in a similar way escape present web pleasure form embarking on a voyage to economy. While being tied to actions of a encounter new sea monsters, to use a Dada person, the profiles don’t represent data that metaphor, such as the generated profiles? relate to a potential buyer since their Perhaps this is the new creative act: data aggregation is randomized. In this gesture, the design and arrangement as opposed to data system temporarily escapes the established creation. The profiles undergo a certain design economic construct and exists only for its own procedure according to a set of rules. In thus sake. created system the content is almost secondary In evading the online market place, the while the logic framing it endows the system gesture reclaims the meaning given to our with an aesthetic form recognizable to the modes of production on the Web. The actions human mind. This notion is congruent with the contributing to the construction of the online popularity of distribution platforms, which often profiles are not representative of an individual make a bigger splash than the content being and consequently cease to have any marketing channeled through them. For example, Tweeter value. Thus, the work performed is rescued is worth fantastically more than any of its single from the limbo positioned in between leisure posts. Similarly, a new online music player will and wage labor. Participating in Hyper- create much more of a buzz and for a longer Generator is play and requires no additional time than a new song that can be heard through effort to just being in a space. One does not it. reproduce oneself on the Web in a form of a more or less accurate profile, which could be

28 bought or sold. Instead, the production results in abstractions created for the sheer joy of reshuffling data. If we are to exercise our leisure time on the Web, let it be just that: a play outside of the tedious economies of labor and production. The intensified world can then return to being an expanded playground unaffected by the economic struggle. History, however, teaches us otherwise. In the end, the Avant-Garde gestures become reconciled with the powers, which have been previously evaded. The hegemonic system incorporates the rebellion as part of itself. Regardless, the fun of inventing new spaces for play continues.

References

Andre Breton. 1948. Catalogue of an exhibition at La Dragonne, Galerie Nina Dausset, Paris, Le Cadavre Exquis: Son Exaltation, pages 5-7. Galerie Nina Dausset, Paris.

Sandra Harding. 1996. From Feminist Empiricism to Feminist Standpoint Epistemologies. In Lawrence Cahoone, editor, From Modernism to Postmodernism, an Anthology, page 630. Blackwell Publishers, Oxford.

Karl Marx. 1977. Economic and Philosophical Manuscripts. In David McLellan, editor, Karl Marx: Selected Writings, page 76. Oxford University Press, Oxford.

!""#$%!&'(")'*+,-,*%"#./0&!$'"1230$',/$%!*-4"5'!

Abstract *)OO$"E%U/"E %)%$Q*"4)/)!"E% $ $%&'%Y&I%)-% ,4)/"!ID%!%V) % &+$!*"E% "+"4)/%!&%!*" %!*)!%-/&H$% %!*" %$ )I%" %O/$ $!$-%!*$%$+$/E$,$%&'% # % !&% V/"!$ % !*" % $ )I% /$4)!$-% !&% ) % U4& &+"E% PJQO$/"+$!)4R%7&+O#!$/%./!%"%@/)A"4;%"! %*" !&/",% $QO$/"+$!)4 % 7&+O#!$/ % ./! % " % @/)A"4D% . % )/!% ,&!$Q!#)4"A)!"&%)-%E$$/)4%-$H$4&O+$! D%%S*" % /$4)!$-%V"!*% $ &/I%$QO$/"+$!)4%$QO$/"$,$ Z%)% T"-%&'%)/!;%"++)$!4I%,&$,!$-%!&%,&+O#!$/ ;% $QO$/"+$!)4%)/!;% !/&E;%U&4-;%$)E$/%!&%$QO4&/$%"! % $+O*) "A$ % $ &/I; % O&$!", % )- % )$ !*$!",% !$,*&4&E",)4%$H"/&+$!%) %"! % #OO&/!;%U/$)T"E% $QO$/"$,$ % "!$/C+$-")!$- % UI % ,&+O#!)!"&)4% U&#-)/"$ ;% )OO/&O/")!"E % ,"$!"'", % ,&,$O! ;% -$H",$ ;% V*",*% U$,)+$ % )4 &% !*$ % )/!V&/T % /$!*"T"E%)-%,&!)+")!"E%!*$+%V"!*%O&$!",%)-% #OO&/!W+$-")D%!%" %)%)/!%'&,# $-%&%!*$%!/"4&EIX% )$ !*$!",%H)4#$ D% )/!" ! ;%,&+O#!$/ %)-%O#U4",D%S*$%./!% !"!#!$%)!% !*$ % G"H$/ "!I% &' % @/) "4") % ,/$)!$-% )- % O/&+&!$ % ["/ ! % V$ % V"44 % U$E" % V"!*% ) % U/"$'% 7&+O#!$/ % ./! % ,&#/ $ ;% $Q*"U"!"& % )- % !*$% "!/&-#,!"&%!&%!*$%$+$/E$,$%&'%7&+O#!$/%./!%"% !*$&/$!",)4CO/),!",)4%+$!*&-&4&E"$ %-$ ,/"U$-%*$/$;% @/)A"4;%&+")!"E% &+$%&'%!*$%O"&$$/%)/!" ! %)-% #,*%) %!*$%,&,$O!%&'%+$!)C" !)44)!"&D%0",$%!*$% $H$! D%0$,&-4I;%)%-$ ,/"O!"&%&'%*&V%!*$%/$4)!$-% V$%*$/$%O/&O& $%!&%$Q)+"$%!*" %)/!" !",%O/&-#,!"&% )/! %V&/T %)/$%"!$E/)!$-%"!&% O$,"'", %$Q*"U"!"&% V*",*; % " % &#/ %H"$V;% $QO/$ $ % ) % $+$/E$!% O),$ %UI%)OO4I"E%V*)!%S)")%[/)E)%"!/&-#,$ %) % @/)A"4")%"-$!"!I%"%,&!$+O&/)/I%)/!D% +$!)C" !)44)!"& D% 0&+$ % $Q)+O4$ %)/$% -$ ,/"U$-% )- % "44# !/)!$-D % S*$ % "-$) % &' % -"''$/$! % T"- % &'% ;$(<.'= O),$ %'&/%"&H)!"H$%V&/T %&'%7&+O#!$/%./!%*)H$% @/)A"4") % 7&+O#!$/ %% ./!;% +$!)C" !)44)!"&;% U$$%O/&Y$,!$-%)-%+&#!$-D%.V)/$%&'%!*$%/" TI% G"H$/ "!I%&'%@/) "4") !) T%V$%O/&O& $;%!*" %$ )I%*) %U$E#%!&%-$4"$)!$% !*$ % '"$4-\ % O/&'"4$ % ) % ) % I+U&4", % '$$-U),T% "!&% >%!'.=&?!,.% @/)A"4")%7&+O#!$/%./!D%[")44I;%V$%O&"!%!&% &+$% 0&+$!"+$ %)%'4",T$/"E%!*&#E*!%O&"! %!&% O& "U4$%-$H$4&O+$! %&'% #,*%)/!D% &+$!*"E%!*)!%*) %'4&)!$-% *)O$4$ 4I%"%&#/%+"-% E"H"E%"!%+$)"E;%+)T"E%# %#-$/ !)-%!*)!% #,*% )%)OO)/$!4I%!*&#E*!'#4%+&+$!%*)-%)%,)!*)/!",% "E"'",),$D%% . % ) % U&4! % &' % 4"E*!"E % #,*%

31 @, !.',?*-"#.%!$3!&*-,+*!,.% !%" %"+O&/!)!%!&%O&"!%&#!%!*)!%@/)A"4")% ,&+O#!$/%)/!" ! %*)H$% #''$/$-% !/&E%"'4#$,$%&'% %@/)A"4;% )/&#- %!*$%I$)/ % >NL]; %)% +)44% !*$%@/)A"4")%!*$&/$!",") %_#&!$-%)U&H$%o%+)"4I% E/&#O%&'%)/!" ! %)-% ,"$!" ! %)+$-%'&$ !*$!",)% !*$%)/!%,/"!"_#$%)-%,#/)!&/%./4"-&%(),*)-&;%!*$% M/&#O>;% U$E) % !&% $QO4&/$ % ) % )/! % '&/+ % !*)! % V) % $+"&!",") % jk,") % 0)!)$44);% )- % !*$ % 7A$,*&C $+$/E"E%)!%!*$%!"+$;%"%V*",*% $ &/I;%O&$!",%)-% @/)A"4")%+$-")%O*"4& &O*$/%l"4m+%[4# $/%o%V*&% )$ !*$!",%$QO$/"$,$ %V-%U$%"!$/C+$-")!$-%UI% *)H$ % U$$ % " !"E)!"E % )/!" ! % )U&#! % !*$% ,&+O#!$/ D% S*$ $ % ,&+O#!$/ % ),!$-% ) % -$H",$ % $QO$/"+$!)4%O&!$!")4%&'%%,&+O#!$/%!$,*&4&E"$ D% ,)O)U4$ % &' % /$ O&-"E % !&% O/&,$-#/$ ;% #,*% ) % [4# $/ % 4"H$-% " % 01&% 3)#4&% '&/ % p? % I$)/ D% i" % O/&,$ "E % )- % ) V$/"E % "!$/),!&/ ^B% /$_#$ ! ;% )/!",4$ % "_#"/$-%% )U&#! % !*$ % /&4$ % &' % )/!" ! % )-% )-; % )! % !*$ % )+$ % !"+$;% V$/$ % !*$ % )/!V&/T % O*"4& &O*$/ % " % O& ! % "-# !/")4 % )E$ % "% #OO&/!W+$-")D%0#,*%O/&,$-#/$ %,*)/),!$/"A$-%!*$% ,&!$+O&/)/I% &,"$!I;%O&"!"E%!&%!*$%O& "U"4"!"$ % )/!V&/T % O/&O& $-% )! % !*)! % !"+$ % " % H$/I% O$,"'",% &'%!$,*",)4%&UY$,! p%U$,&+$%`,)//"$/ %&'%+$)"Ea% V)I ;%)-%/$_#"/$-%'/&+%!*$%)/!" ! %O/&E/)++"E% )-%!*$%!/) '&/+)!"&%&'%`O$&O4$%"!&%-$ "E$/ %&'% T&V4$-E$ % !&% `U/$)T% !*$ % ,&-$ ac>dD% S*$"/ % +)"% +$)"E%"%)%O)/!",#4)/%O/&,$ a%cBdD% E&)4 % V) % &! % !*$ % -$H$4&O+$! % )- % /$ $)/,* % &% ,&+O#!$/ % ,"$,$ % )4E&/"!*+ %U#! %!&% ),_#"/$ %!*" % S*$ %)+$;% e#+$/", % 7&+O#!$/ % ./!;%%V) % !IO$%&'%%T&V4$-E$%"%&/-$/%!&%,/$)!$%+$)"E'#4% '/$_#$!4I%_#$ !"&$-;%+& !4I%U$,)# $%)!%!*$%!"+$% $ &/I;%O&$!",%)-%)$ !*$!",%$H"/&+$! D%0#,*% !*$/$%V) %)%$$-%!&%" $/!%,&+O#!$/ %"!&%!*$%)/!% )/! % V) % /$'$//$-% !&; % )! % !*)! % !"+$;% ) % e#+$/",% O/),!",$%"%) % !"#% V)I;%+)IU$%) %$"%&'('#!;%&/% 7&+O#!$/%./!%&/%7IU$/$!",%./!D% ")"*!#+&* ( '#!;% &/ %,"-( '#!;% $H$ %.+!"#+"! ( '#!;% )+&E % &!*$/ % !$/+ D% q!*$/ % @/)A"4") % )/! % E/&#O % S*$%@/)A"4")%E/&#O%-"-%&!%,&+$%#O%V"!*% )4 &%U$E#%!&%V&/T<%V"!*%,&+O#!$/ ;%U#!% &+$%&'% !*& $%!$/+ %&/%-$'""!"& D%%S*$I%V$/$%)4/$)-I%"% !*$+%-"-%&!%)OO/&),*%"!^ %" $/!"&%"% #,*%)% !/",!% # $%UI%)/!" ! % #,*%) %@$/)/-%7)"44)#-%P[/),$R;% $ $=;%+$)"E%!*)!%!*$%O/&,$-#/$ %)-%O/&,$ $ % f)4-$+)/%7&/-$"/&%P@/)A"4R;%%g& *"/&%h)V)E#,*"% &' % ,&+O#!"E % V$/$ % $QO4&/$-% ,&,$O!#)44I; % &/% P:)O)R %3)#4 % @/&V % PGhR;% i$U$/! % [/)T$% " !/#+$!)44I; % U#! % &! % )4V)I % "++)$!4I % UI% PM$/+)IR;% e",*&4) % 0,*&$''$/ % Pi#E)/IR;%:&*% E&"E%)'!$/%!*$%\U/$)T"E%&'%,&-$ \%!*$+ $4H$ D% f*"!$I%PG0.R;%j"4")%0,*V)/!A %PG0.R;%[/"$-$/% e)T$ %% PM$/+)IR;% !&% _#&!$ % Y# ! % ) % '$VD% % !*" % $ )I% V$ % ) #+$ % !*$ % !$/+% S*$&/$!",") % #,*%) %./4"-&%(),*)-&%P@/)A"4R;% #./0&!$'"5'!% !&%-$!$/+"$%!*$% O$,"'",%)/!" !",% jk,") % 0)!)$44) % P@/)A"4R;%l"4m+ % [4# $/% O/&-#,!"&%"%-" ,# "&D%f$%,& "-$/% #,*%)/!%) % P7A$,*& 4&H)T")C@/)A"4R;%3*"4"O$%n#$)#%P[/),$R;% "++)$!4I% ,&$,!$-% !&% ,&+O#!$/ ;% V*",*% )/$% J-+&#- % 7&#,*&! % P[/),$R;%5&I % . ,&!!% PGhR;% ),!"H$%)E$! %"%!*$%'&44&V"E%!/"4&EIX% )+&E%&!*$/ ;%*)-%)4 &%V/"!!$%)U&#!%!*" %$+$/E$!% >D ./!" ! ;% !*$ % &$ % V*& % ,&,$"H$ % '"$4- % &'% )/!%'&/+D% ),!"& % ,/$)!"E % ,# !&+"A$-% &'!V)/$ % )-% *)/-V)/$;% )- % $H"/&+$! % V*$/$ % $ &/I;%

32 O&$" ! , % )- % )$ !*$!", % $QO$/"+$! % *)OO$ % ) % )/!" ! D%i$% )"-%&%!*)!%&,,) "&%!*)! %`"% @/)A"4;% $QO/$ "H$;%)''$,!"H$%)-%,&E"!"H$%$H$! Z 7&+O#!$/%./! %'&#-%+$!*&-&4&E",)4 % U),TE/&#-% " % 7&,/$!$ % ./!a % cKdD% % >N]B; % *$ % ,/$)!$-% !*$% BD 7&+O#!$/ %P&/%)%$H"/&+$!%&'%+",/&,*"O ;% ./!$s",) % 7$!/$% )! % !*$ % ./! % !"!#!$ % &' % !*$% $ &/ % )- % ),!#)!&/ R;% !*)! % O/&,$ % -)!)% G"H$/ "!I% &' % 7)+O") ;% Ge7.(3 %c]d;% V*",*% ) V$/"E % /$_#$ ! % )- % '"4!$/"E % ),!"& % *) % *)-% ) % O"&$$/ % /&4$ % '&/ % !*$ % -$H$4&O+$! % &'% ),,&/-"E%!&%!*$"/%O/&E/)++"EZ% )4E&/"!*+",%)/!%/$ $)/,*%"%"+)E$%E$$/)!"&%cLdD pD S*$ % O#U4",;% -$ "E)!$-% ) % "!$/),!&/ ;% V*&% %>NN<;%)'!$/%+#,*% !/#EE4$;%!*$ $%"-$) % )/$%!*$%&$ %,)# "E%,*)E$ %"!&%!*$%,&+O#!$/ % U$E# % !&% U$ % +&/$ %E$$/)44I% ),,$O!$-%UI% !*$% $H"/&+$! ;% O/&H&T"E % ) V$/ % !&% #,*% @/)A"4")%)/!%,&++#"!I%)-%!*$%'"/ !%@/)A"4")%)/!% ),!"& D% E/)-#)!"& % O/&E/)+;% '&,# "E % !*$ $ % " #$ ;% V) % S*" % H" ,$/)4 % !/"4&EI% " % "++)$! % !&% !*$% ,/$)!$-%)! %!*$%./! % !"!#!$%&'%!*$%G"H$/ "!I%&'% &!&4&E",)4%)-%$O" !$+&4&E",)4%7&+O#!$/%./!% $!% @/) "4");%"%@/) "4");%,)O"!)4%&'%@/)A"4D%q!*$/%)/!" ! % #OD #,*% ) % M"4U$/!!&% 3/)-&; % .-/m % 3)/$!$;% ("4!&% ! % " % )4 &% "+O&/!)! % !&% !)!$ % !*)!% 0&E)U$%)-%6"))%6&+"E#$ %)4 &%),,$O!$-%!*$% `$QO$/"+$!)4%)/!%*) %*)-%)% !/&E%/&4$%"%@/)A"4% ,*)44$E$ % )- % ) % '$V %I$)/ % 4)!$/ % V$/$ % )4 &% +)IU$ % U$,)# $ % !*$ % )/! % +)/T$! % -"-% &! % &''$/% ,&&/-")!"E % /$ $)/,*$ % " % 7&+O#!$/ % ./! % )! % !*$% "E"'",)! % O& "U"4"!"$ % '&/ % )/!" ! % !*)! % -"-% &!% !)!$ % &' % 01&% 3)#4&; % 5"&% -$ % :)$"/&% )- % 5"&% '&44&V % !*$ % ,#//$! % O)/)-"E+ D% 0&+$ % &' % !*$ $% M/)-$%-&%0#4D% $QO$/"+$!)4%)/!" ! %V$/$% &%/)-",)4%!*)!%!*$I%V$/$;% %>NN=;%[/)E)%O/$ $!$-%*$/%3*6%/$ $)/,*% -#/"E % !*$"/ % 4"'$;% !/&E4I% /$Y$,!$- % UI % !*$% V"!*% !*$ % $Q*"U"!"& %/!"#"*0&* ( /&$1)'!&+% )!% +)" !/$)+%f$ !$/%)/!D%[&/%$Q)+O4$;%!*$%!&-)I% !*$%01&%3)#4&%(# $#+%&'%+)E$%)-%0&#-%)-%"% ",$ $-%)/!" !%jIE")%74)/TK%V) %)4+& !%$QO#E$-% >NNK % *$ % &/E)"A$-% !*$ % '"/ ! % &4"$ % !$/$!% '/&+%0&/U&$;%"%[/),$;%"%!*$%]?r ;%U$,)# $% *$% $Q*"U"!"&%'&/%!*$%!$/)!"&)4%l" #)4%0$+"&!", % -)/$-%!&%$QO$/"+$!%V"!*%H" ,$/)4%U&-I% $ )!"& % 7&E/$ %*$4-%)!%%01&%3)#4&%7)!*&4",%G"H$/ "!I;% )- % '$$4"E % # "E % &/-")/I% +)!$/")4 % #,*% ) % 3G7C03D% % >NN=; % 6&+"E#$ % ,#/)!$-% !*$% &"& % T" ;%O$UU4$ ;%O4) !",%U)E ;%)+&E%&!*$/ ;% $Q*"U"!"&%2#!( (!3"(44.(5"+!1#6%)!%!*$%(# $#+% *)H"E%!*$%O)/!","O)! %U&-"$ %) % #OO&/! %'&/%*$/% &' % 7&!$+O&/)/I% ./! % &' % !*$ % G"H$/ "!I% &' % 01&% V&/Ta%%cpdD% 3)#4&% )- % " % >NNN % ) % O&/!)4 % )! % !*$ % ($/,& #4% S*$ % O"&$$/ % )/!" ! % f)4-$+)/ % 7&/-$"/&% @"$")4%"%5"&%M/)-$%-&%0#4D%%>NN];%6)"$4)% U$E)%!&%# $%,&+O#!$/ %"%H" #)4%)/! %"%@/)A"4%UI% @&# &%,#/)!$-%!*$%$Q*"U"!"& %7"%&'8:"% )!%!)#% !*$%$-%&'%!*$%K?\ ]%cN]>;%*$% *&V$-%*" %,&+O#!$/%)/!V&/T %)!%!*$% -&+)"%"%&/-$/%!&%)44&V%"!%!&%-")4&E#$%V"!*%!*$% $Q*"U"!"&%./!$s",)%c=d%)-%O/$ $!$-%)%+)"'$ !% O#U4",a%cNdD%j)!!$/%&;%) %-"/$,!&/%&'%!*$%(# $#+% V"!*%!*$% )+$%)+$ %"'4#$,"E%+)I%@/)A"4")% &'%+)E$%)-%0&#-% *$%,/$)!$-%!*$%'"/ !%@/)A"4")%

33 +% $-") % 4)U )- % " !"!#!$-; % '&/ % !*$ % '"/ ! % !"+$ % "% V*& % # $-% !&% V&/T% &4I% V"!*% l"-$&% ./! % *) % @/)A"4;%)%)/!" !C"C5$ "-$,$%O/&E/)+%'&/%I&#E% &/E)"A$-% $Q*"U"!"& % ",4#-"E % 7&+O#!$/ % ./!% )/!" ! D " !)44)!"& D%

0",$ % >NN]; % !*$ % @/)A"4") % !)#% 7#4!#/)4% A 9$!*B,% !*--*!,.% C" !&=(B?* $ !"!#!$ % *) % *)-% ) % 4$)-"E % /&4$ % " % !*$ % '"$4-% &'% ($)V*"4$;%)!%!*$%G"H$/ "!I%&'%@/) "4");% 7&+O#!$/%./!D%!%*) %U$$% *&V"E%)-%)V)/-"E% V*)! % # $-% !&% U$% ) % +)44 % U")#)4 % +$$!"E % V) % )/!" ! %V&/T"E%V"!*%)/!%)-%!$,*&4&EI%)-%*) % !/) '&/+$-%"%)%)#)4 %!$/)!"&)4 %7&E/$ % O/&+&!$-% H$/I% "+O&/!)! % "!$/)!"&)4 % )/! % )-% )4V)I %*$4-%V"!*%)%7&+O#!$/%./!%$Q*"U"!"&D%S*$% !$,*&4&EI%U"$")4 %$Q*"U"!"& %)+$-%2#!< &*&')( !*$&/$!",)4 % O/),!",)4 %+$!*&-&4&E"$ % -$H$4&O$-%!&% =$!&+D%()I%)/!" ! %_#&!$-%"%!*" %$ )I%*)H$% $! % #O % #,*% $Q*"U"!"& % )- % #U $_#$! % /$ #4! % V&%)V)/- %)-%,&++" "& %'/&+%!*" %" !"!#!$D%% '/&+%!*$ $%$QO$/"$,$ %V$/$%!*$%!/"EE$/ %'&/%!*" % . % $,&- % E$$/)!"& % &' % )/!" ! ;% V&/T"E% $ )ID% V"!*% )/! % )- % !$,*&4&EI; % V) % /" "E % )/&#- % !*$% % B??<; % -#/"E % &$ % !*$ $ % )#)4% I$)/ % >NNKCB??LX % .) % @)//& ;% 7)/4& % 3/)#-$;% ,&E/$ $ % _#&!$-% )U&H$;% V*",*% ,&++$+&/)!$-% 74$&+)/ % 5&,*);% 6)"$4) % h#! ,*)!;% 6&#E4) % -$% !$%I$)/ %&'%!*$%G"H$/ "!I%&'%@/) "4")%E/)-#)!$%)/!% 3)#4);% 6#4,"+"/) % 7)O" )";% J-E)/ % [/),&;% O/&E/)+;%!*$%$Q*"U"!"&%>?@A%PB#"'!"#(#(=C1')( J#'/) "&% 3/)!$ ;% [/)," ,&% ()/"*&; % M#!&% ! ( @AR % V) % *$4-% )! % @)T % &' % @/)A"4^ % 7#4!#/)4% euU/$E);%M" $4$ %@$"4E+); %H)"%0)!));%h8!")% 7$!/$%"%@/) "4")D%[&/%!*)!%&,,) "&%S)")%[/)E)% (),"$4;% j#,) % @)+U&AA";% jk,") % j$1&; % j#" )% )- % f)E$/ % @)/Y) % ,#/)!$-% !*$ % *&V % )- % [/)E)% 3)/)E#)";% j#"A % 6#H);% jIE") % 0)U&");% ()/,& % "!/&-#,$-%!*$%,&,$O!%&'%+$!)C" !)44)!"&;%V*$/$% @) !& ;% ()/") % j#"A) % P()4#R % [/)E& &; % ()/!*)% !*$%'&,# %&'%)4+& !%&'%)44%V&/T %V) %"!$/),!"H"!ID% M)U/"$4;% 5)_#$4 % h&E); % 5)_#$4 % v#)&; % 5$Y)$% S*$ % "-$) % V) % !&% ,/$)!$ % O),$C!"+$ % -")4&E#$ % 7)!&";% 5& )E$4) % j$&!$;% 0)-/&% 7))H$AA";% )+&E%!*$ $%)/!V&/T ;%)-%&!%)% $!%&'% $O)/)!$-% 0"4H") % j)#/$!" ;% 0"+&$ % (",*$4"; % l)4 $4"% "-$O$-$! % " !)44)!"& D% S*$ % +$!)C" !)44)!"&% 0)+O)"&; % f"4!& % .A$H$-&; % )+&E % +)I% &!*$/ D% O/&Y$,! % '&/ %>?@A% &! % &4I% E)H$ % ) % &/E)",% 0&+$%&'%!*$+%)/$%&!% O$,"'",)44I%\,&-$%U/$)T$/ \% ,&*$/$,$%!&%!*$%$Q*"U"!"&^ % O),$%U#!%"!$E/)!$-% U#! % $QO4&/$ % ,&+O#!$/ % I !$+ % " % ,&44)U&/)!"&% !*$%)/!%O"$,$ %$QO4&/"E%!*$"/%O&$!",%O/&O& )4D%S*$% V"!*%O/&E/)++$/ D% )/,*"!$,!#/$ % &' % !*$ %>?@A% $Q*"U"!"& % V) % S*$&/$!",") %3/" ,"4)%./)!$ %)-%(&",)% ,&,$"H$-% +"//&/"E % H"/!#)4 % /$)4"!I% !$,*&4&EI% S)H)/$ ;% ./! % i" !&/") % e)/) % 7/" !") % 0)!& ;% )H"E)!"&\ % +&-$4 D% `S*$ % +$!)C" !)44)!"&% ,#/)!&/ %[/),"$44$%["4"O"";%f)E$/%@)/Y);%3)#4)% ,& " !$-% &' % V&/T %UI% )/!" ! % )- % /$ $)/,*$/ % 3$/" "&!&; % -),$ % ,#/)!&/ % ()"/) % 0O)E$/&;% V&/T"E%+& !4I%V"!*%!*" %!IO$%&'%)/!D%S*$ $%V&/T % '&44&V"E % (),*)-&; % @&# &% )- % 0)!)$44)\ % U/$),*% !*$ % )/,*"!$,!#/)4 % O),$C!"+$ % !/#,!#/$;% 4$)-$/ *"O;%*)H$%U$E#%!&%"H$ !"E)!$%!*" % O$,"'",% ,/$)!"E%!#$4 %&'%$H$! %!*)!%)44&V%!*$%O#U4",%!&% !IO$%&'%)/!D%JH$%,#/)!&/ % #,*%) %7*/" !"$%($44&% -"H$%"!&%!*$"/%,&!$!Z%!*$I%$ #/$%!*$%&C4"$)/%

34 )!#/$%&'%!*$%/&#!$ %!*/&#E*%!*$%+$!)C" !)44)!"&;% [/)E& &; % 0"4H"&% v)+U&";% 0#A$!$ % l$!#/$44" % )-% /$,&'"E#/"E % !*$ % )/,*"!$,!#/)4 % O),$ % &' % !*$% S)") % [/)E)D% S*$ $ % )/!" ! % !)- % &#! % '&/ % !*$"/% E)44$/"$ ;%/$C-"/$,!"E%!*$%E)A$%)-%!*$%+&-$ %&'% "&H)!"H$%)-%&/"E")4%O/&O& )4 ;%+)"4I%"%!$/+ % O$/,$O!"&%&'%!*& $%+&H"E%!*/&#E*%"!a%c>?dD &'%$QO4&/"E%!*$%O&!$!")4%&'%,&+O#!$/%4)E#)E$ ;% $Q!/),!"E%)-%-$O4&I"E%$V%O&$!", D% S*$/$'&/$;% !*$ % !/#,!#/)4 % &/E)"A)!"& % &'% !*$%$Q*"U"!"&%$H"/&+$!%V) %-$ "E$-%!&% !/$!,*% 7&+O#!$/%4)E#)E$ %V$/$%&!%,/$)!$-%'&/% !*$ % ,&,$O! % &' % " !)44)!"&; % E$$/)44I% # $-% '&/% )/!" !",%$QO/$ "&;%U#!% &+$%)/!" ! %*)H$% #UH$/!$-% ,&!$+O&/)/I% V&/T % &' % )/!;% V*$/$ % $),*% )/!" !% !*$"/ % # $;% )44&V"E % !*$ % )OO$)/),$ % &' % $V % )/!% O/&-#,$ % )- % &/E)"A$ % *" % &/ % *$/ % +",/&% O),$% '&/+ D% i#+) % $ &/")4 % )O)/)!# % $)U4$ % "-"H"-#)44ID%S*" %&/E)"A)!"&%)"+$-%!&%#H$"4%!*$% \)" !*w y)\N%D%f*$%)/!" ! %,/$)!$%-"''$/$!%V)I %&'% "E"'",)!%) O$,! %!*)!%V$/$%U$"E%!/) %,&-"'"$-% O$/,$"H"E % !*$ % V&/4-; % "!$/+$-")!$-%UI% "!&% $ )!"& ;%V*",*%,-%U$%'$4!%)-% *)/$-D%`% ,&+O#!)!"&)4 % -$H",$ ;% !*$I% )/$ % /$C"H$!"E% E$$/)4;% !*$ % V&/T % & % -" O4)I% ,4# !$/ % "+)E$ ;% \)" !*w y)\D%($!)C" !)44)!"&\ %-$ "E%O/&H"-$-%!*" % &#- ;%!),!"4$% $ )!"& %)-%+&H$+$! ;%) %!*$% $QO$/"$,$%,& $_#$!4I%O/&+&!"E%)$ !*$!", D% &#!,&+$ % &' % ,&44)U&/)!"H$ % O/&,$ $ % )+&E % !*$% S*$ % V&/T % -" O4)I$-% )! % !*$ % $Q*"U"!"&% )/!" !;% !*$ % O#U4",;% !*$ % +),*"$ % )- % !*$ % +)I% /$O/$ $!$-% )- % "!$E/)!$-% U&#-)/"$ ;% +"- %V*& $%"!$44"E$,$%" %$+U$--$-%"%!*$+aD% )+)4E)+)!"E%+# ",;%) %V$44%) %H" #)4%)-% ,$",% i#+)%)-%)/!"'",")4 %"!$44"E$,$%V$/$%#"!$-%!&% )/! ;% V"!*% ) % U/&)-C/)E$ % &' % ,"$!"'", % )/$) D% @I% ,/$)!$ % ) % I+U"&!", % ,&++#"& % )+&E % !*$% *&V,) "E%!*" %!IO$%&'%O/&-#,!"&%!*" %$Q*"U"!"&% "!$/),!&/ \ % +"-; % )/!" ! % )- % +),*"$ D% S*$% H)4#$-% !*$ % O"&$$/ % V*& % '),$-% #"+)E")U4$% $Q*"U"!"&%V) %*$4-%"%!V&%E)44$/"$ D%S*$%`!#$4 % -"''",#4!"$ % " % -$H$4&O"E % !*$"/ % &#!O#!;% O4),"E% &' % $H$! % " % !*$ % #OO$/ %E)44$/I% V)-$/ % !*/&#E*% @/)A"4%)-%@/) "4")%"%)%4$)-"E%O& "!"&%V"!*"%!*$% '"$4- % &' % O& "U"4"!"$ % ,&!)"$-% V"!*" % !*$% "!$/)!"&)4%7&+O#!$/%./!%,&++#"!I I!*$ " % &' % "+)E$ ;% &#- ;% )U !/),! % ,&,$O! ;% @$44&V;%["E#/$ %>;%B;%p%)-%<;%)% $_#$,$% !),!"4$ % $ )!"& % )- % O& "U4$ % " &+&/O*" + % &'%"+)E$ %&'%!*$%%>?@A(O/&Y$,!D )+&E % !*$ $ % ,)!$E&/"$ D% PDDDRD% S*$ % 4&V$/ % E)44$/I% &''$/ % ) % *IU/"-% !$44#/", % '"$4-% V*$/$ % "+)E$ % &'% I!*$ " % )- % +)!$/")4 % ,#4!#/$ % U4$-; % U/$)T"E% !*/&#E*%!*$%U)//"$/ %&'%V*)!%" %&/+)44I%,)44$-%!*$% /$)4a%c>>dD

S*$ % )/!" ! % )! % !*" % $Q*"U"!"& % V$/$X % @")% ($-$"/& ;%7*",&%()/"*&;%6)"$4)%h#! ,*)!%)-% 5$Y)$ % 7)!&";% 6")) % 6&+"E#$ ;% M"4U$/!!&% 3/)-&; % j#" ) % 3)/)E#)";% jIE") % 0)Uu");% ()/E)/"!)% 0,*#4!A % P,#/)!&/ % O/$ $!"E % !*$ % V&/T% &' % )% ["E#/$%>X%GOO$/%H"$V%&'%!*$%'"/ !%'4&&/ ,&44)U&/)!"H$ % j)!" % .+$/",) % E/&#OLR;% ()4#%

35 S*$I%*)H$%U$$%,&,$"H$-%)-%O/&-#,$-%V"!*%!*$% #OO&/!%&'%!*$%./!% !"!#!$%&'%!*$%G"H$/ "!I%&'% @/) "4") % )- % !*$ % @/) "4") % 5$O#U4",^ % e)!"&)4% (# $#+D%

S*$%,&,$O!%&'%+$!)C" !)44)!"&%*) %U$$% )OO4"$-%'&/%!*$ $%$Q*"U"!"&% O),$ %"% #,*%V)I% !*)! % !*$ % O/&O& $-% $! % #O % *)H$ % )44&V$-% !*$% $+$/E$,$%&'%$V%+$)"E %/$ #4!"E%'/&+% O),$C !$+O&/)4 % /$4)!"& % ,/$)!$-% )+&E % )/! % O"$,$ ;% /$% ["E#/$%BX%0"+#4)!"&D(0"+$!/") %PjIE")%0)U&")R% "E"'I"E % !*$ % &/"E")4 % O/&O& )4 % &' % !*$ % )/!" ! % &/% )-%qOC$/)%P6)"$4)%h#! *,*)!%)-%5$Y)$% )/!" !^ %E/&#O >>D%S*$%,#/)!&/")4%)OO/&),*%!&%!*$ $% 7)!&"R%% $Q*"U"!"& %U$E" %UI%-$'""E%!*$%O*I ",)4% O),$% &'%!*$%$Q*"U"!"&;% &%!&% )I;%)44%)/$) %-$ "E)!$-%'&/% !*$ % )/!V&/T ;% V*",*% V"44 % *$4O% ,/$)!$ % )% )/,*"!$,!#/)4%-$ "E%!*)!%,& $_#$!4I%O/&+&!$ %)% &/!%&'% $4'%&/E)"A)!"&%'&/%!*$%+&!)E$D%.4 &;%!*$% O/&,$ %&'% $4$,!"E%!*$%)/!V&/T %'&/%!*$%$Q*"U"!"&% *) % U$$ % /$4)!$-% !&% !*$ % ,&,$O! % &' % +$!)C ["E#/$%pX%j)I%&#!%&'% "Q%-"''$/$!%)/!V&/T " !)44)!"& %)-% $4'C&/E)"A)!"&D %%!*$%,) $%&'% !*$ %=$7"& ( *&V % !*/$$ % )/$) % *)H$ % U$$% )H)"4)U4$X%

>D -&&/ % +# $#+ % " !)44)!"&^ % O),$ % '&/% )#-"&H" #)4%)/!V&/T %V"!*%)-%V"!*&#!%!$/$!% ,&$,!"&Z

BD JQ!$/)4% O),$;%&#! "-$%%!*$%(# $#+%P)%U"E% V*"!$%,#O&4)%'&/%"E*!%O/&Y$,!"&%O/$ $!)!"& RZ%

pD SV&% )#-"!&/"#+ ;% # $-% '&/ % ,&,$/! ;% O$/'&/+),$ ;%+# ",%)-%l: % *&V D% ["E#/$%?;% V$%O/$ $!%&$'F"('/&+%!*$ $%$Q*"U"!"&< 0",$%B??L>?;%)% $!%&'%$Q*"U"!"& %)+$-% =$7"& (*)H$%E&$%'&/V)/-%V"!*%O/&Y$,! %*)H"E% "+"4)/%)OO/&),*%!&%>?@AD%=$7"&% " %!/) 4)!$-% 4"!$/)44I%) %.+E"!,""+("%3&/!#E#$ $(U#!%$"&%)4 &% +$) %$"%&'%%,/$)!"E%)%O4)I'#4%E)+$%&'%V&/- D%

36 ["E#/$%LX%=$7"&GJ;% $!%#O%-$ "E;%B?>>

["E#/$%=X%=$7"&GH;%$Q*"U"!"&%$!/I%*)44;%B??L

["E#/$%NX%=$7"&GJI% $!%#O%-$ "E;%B?>>

["E#/$%KX%=$7"&GH;%" !)44)!"& ;%B??L ["E#/$%>?X%=$7"&GJI%O)&/)+",%H"$V;%B?>>

=$7"&K % ,#/)!&/")4 % )OO/&),*% -$ "/$ % !&% U/"E % !&E$!*$/ % &+$ % &' % !*$ % +)" % ) O$,! % "% 7&+O#!$/%./!%/$ $)/,*%)-%O/&-#,!"&%"%@/)A"4;% V"!*%O/&Y$,! % #,*%) X

> ./!V&/T % # "E % ,&+O#!$/ % ) % !&&4 % !&% ,/$)!$ %% "+)E$ % &' % I!*$ " ;% &#- ;% O*&!&E/)O*", % )- % H"-$&CE/)O*", % "+)E$ ;% ["E#/$%]X%=$7"&GHI%O$/'&/+),$%UI%J#'/) "&% "!$/),!"H$%&/%&!Z 3/)!$ ;%B??L B ./!V&/T % ,&,$"H$-% !&% U$ % U/&)-,) !% !*/&#E*%!*$%!$/$!Z p qO$ % O),$ % '&/ % O)/!","O)!"& % )- % $4'C &/E)"A)!"& % # "E % H"-$&% ,&'$/$,$% &'!V)/$D%

37 < (# ", % *&V ; % O$/'&/+),$ % )- % l: % ,/$)!"E%O& "U4$%&/E)"A)!"& %!*)!%+)I%&!%*)H$% *)H"E % ,&+O#!$/ % )/! % ) % U),TE/&#- % '&/% U$$%O/$-",!$-%UI%!*$%)/!" !CO/&E/)++$/D%!%" %) %"'% !*$"/%V&/TD !*$%"!$/),!&/%" %)44&V$-%!&%!&%O4)I%)-%$QO$/"+$!% V"!*%)%'""!$% $!%V*",*%*) %4"+"!4$ %U&#-)/"$ D% D 5"E, ,.%*'("F'$*/G S*$/$'&/$%7&+O#!$/%./!%U$,&+$ %+#,*%+&/$%!*)% S*$ % O/$ $! % $ )I% )"+ % !&% U$E" % )% 7&+U")!&/")4%./!D%!%U$,&+$ %H"/!#)4%/$)4"!"$ D% /$'4$,!"&%&%@/)A"4")%$QO$/"+$!)4%7&+O#!$/%./!% . % "44# &/I% $ $ % "+O/$ "& % )-% )- % )/!" ! \ % O/&,$ $ % O&"!"E % !&% ) % '"$4-% &'% /$O/$ $!)!"& %"!%O&"! %!&%)%,4) ",)4%)/!%!/)-"!"&% -"''$/$! % O& "U"4"!"$ % '&/ % )/!V&/T % O/&-#,$-% V*"4$%E&"E%)'!$/%O/&E/)++$-%),!"& ;%O$/,$O!"& % !*/&#E*%,&+O#!$/%,&-"ED%f*)!%" %!*$%-"''$/$,$% )- % $ )!"& % "! % )OO/&),*$ % 7& !/#,!"H" +;% U$!V$$ % !*$+% )- % &!*$/ % +$) % &' % ,/$)!"E % )/!{% 7&,/$!" +%)-%(""+)4" +%)/!%+&H$+$! %c>=dD% 0$$T"E%!*" %-"''$/$!")!"&%-&$ %&!%"!$-$-%!&% f"!*" % #,*% ,&!$Q! % &!"& % #,*% ) % /$)4"!I;% $*),$%!*$%H)4#$%&'%&$%V*"4$%-$O/$,")!"E%!*$% H"/!#)4"!I; % H" "U"4"!I% c>Kd;% )- % )''$,! % +# ! % U$% &!*$/;%U#!%/)!*$/%!&%#-$/ !)-%!*$% "E#4)/%) O$,! % /$H" $-%c>]dD !*)!%,*)/),!$/"A$%!*$+%c>BdD S*$%H"/!#)4%V&/4- %)/$%)/U"!/)/I%,/$)!"& % .! % ) % !"+$ % 4"T$ % !*$ % O/$ $!;% $QO$/"$,$% &' % *#+) % +"- % )- % !*$ % "!$/),!&/ % )!!/"U#!$ % )% 4$)- % # % !&% $$T% ) % U$!!$/ % #-$/ !)-"E % &' % !*$% +$)"E%!&%!*$+%"% !*$"/%&V%+"- %$H&T"E%)% '&#-)!"& % )- % O& "U"4"!"$ % &' % 7&+O#!$/ % ./!% 4)UI/"!*%&'%/$O/$ $!)!"& D%S*$%)!!/"U#!"&%&'%)% 4&&T"E%'&/%!*$%,&H$/E$,$ %)-%-"H$/E$,$ %"!% \/$)4\% $QO$/"$,$ % !&%)%H"/!#)4 % &UY$,! %,/$)!$ %H$/I% O/&O"!")!$ %V"!*%&!*$/%)/!%'&/+ %c>pdD%./!%"%E$$/)4% "!$/$ !"E % )+U"E#"!"$ % )- % )4 &% )44&V % ) % 4&! % &'% -$)4 %V"!*%$QO/$ "H$%O&$!",% $ $%$QO$/"$,$ %)-% +" #-$/ !)-"E D% &%-&$ %7&+O#!$/%./!D%@#! %7&+O#!$/%./!%-$)4 % S*$ % $!!"E % &' %H"/!#)4 % &UY$,! % )- % !*$% V"!*% +)!*$+)!",)4 % ,&,$O! % " % &/-$/ % !&% ),*"$H$% \$Q" !$,$\%&'%H"/!#)4%V&/4- %$+$/E$%'/&+%!*$% $ $% O&$!", % $QO/$ "& % V"!*" % !*$ % O$/,$O!"H$ % '"$4-% &'% "+O/$ "& %!*$I%O/&-#,$%) % #UY$,!"H$% $ )!"& % *#+) % $ $ % )- % )''$,!"H$ % $QO$/"$,$ D% (&/$% ",$%!*$%O$/,$O!"& %/$ #4!%'/&+%"44# "& D%S*$ $% O$,"'",)44I%!$/),!"H$%7&+O#!$/ %./! %-$)4 %V"!*% #UY$,!"H$% $ )!"& %)/$%&!*"E%+&/$%!*)%+$!)4% /$ O& "H$%)''$,!"H$%)-% $ &/I%'"$4- %,&!)""E% ,&$,!"& D% ()IU$ % !*$ $ % ,&$,!"& % )/$ % !*$% $! %&'% $ $%$QO$/"$,$ %"!$/O/$!$-%) % "E %)-% U&#- % !*)! % $!V"$ % +)!*$+)!", % )- % )/!D% S*$% $QO/$ $-% ) % O/&,$ $ D% S*$ $ % $! % $ !)U4" *% $ $%$QO$/"$,$%!*$%"!$/),!&/%$QO$/"+$! %)-%"! % -"''$/$! % /$4)!"& % )44&V"E % +)I % -"''$/$!")!$-% #UY$,!"H$ % $ )!"& % )/$ % '),! % 4$)H"E % &$% /$)-"E %"%-"''$/$!%+&+$! %&'%!*$"/%/$ O& "H$% E$$/)44I% " % )V$D% S*$ % ,&+O/$*$ "& % &' % !*" % /$O$/!&"/$ D% 5$O$/!&"/$ % V*& $ % T)4$"-& ,&O",% $+$/E"E% $ &/I%&/-$/%/$%"H$!"E%\)" !*w y)\;%) % ,)O),"!I% &O$ % !*$+ % !&% #,,$ "H$ % )- % H)/"$-% !*$%/$ #4!%&'%#+$/",%/$4)!"& %)-%'#,!"& ;%" %)% "!$/O/$!)!"& %c>

38 S*$% #OO&/!%'&/%,&+O#!$/%)/!V&/T %" %)4 &% UI % !*$ % JH$/$!! % "!$/O/$!)!"& % &' % _#)!#+% +)!$/")4 % U#! % !*$"/ % !$,*&4&E",)4 % $!!"E % )/$% +$,*)", a % *&V"E % !*/&#E* % O/),!",)4 % $Q)+O4$ % ,*)/),!$/"A$-%UI%)% I !$+%&'%)OO4",)!"& %!*)!%V"44% `*&V%!*" %$V%+&-$4%,-%%E$$/)!$%p6%E)+$ % O$/'&/+ % !*$ % V&/TD % S*" % I !$+ % !/) C,&-"'"$ % !*)!%)-)O!%)#!&+)!",)44I%!&%# $/% O$,"'",% T"44 ;%)-% "E)4 %"!&% #UY$,!"H$% $ )!"& %# "E%#+$/",)4;% *&V%!*" %$V%+&-$4%,-%'),"4"!)!$%!*$%,/$)!"&%&'% 4&E",)4 % )- % I+U&4", % 4)E#)E$ D% S*$ % /$ #4!"E% !/#4I%"!$/),!"H$% !&/I%4"$ a%c>NdD V&/T%&'%)/!%" %!*$%,&+U"$-%O/&-#,!%&'%!) T %,&-$-% % 0&#!*% .+$/",) % V$ % )4 &% $$ % !*$% !*/&#E*%!*$ $%4)E#)E$ %)-% !&/$-%"%,&+O#!$/% )V)T$"E%&'% #,*%)OO/&),*%"%./E$!");%7*"4$% /$,&/- D% `' % V$ % V" *% !&% '$$4 % !*$ % !$Q!#/$ % &' % )% )- % 7&4&+U") % #-$/ % !*$ % 4$)-$/ *"O% &' % :&/E$ % 4)% O)"!"E % UI % 3&44&,T; % V$ % V-% -)+)E$ % !*$% [$/4);%5",)/-&%6)4%[)//);%()/E)/"!)%0,*#4A;%%4"))% O)"!"E;%) %"!%V) %&!%,/$)!$-%'&/%!*" %O#/O& $D%.% i$/)-$A%M)/,")%)-%5)k4%e"|&%@$/)4;%!&%_#&!$% 3&44&,T% V&/T"E % !*/&#E* % ,&+O#!$/ % ,&-$ % " % )% Y# !%)%'$VD% !$,*&4&E",)4% $!!"E%,/$)!$-%'&/%!*" %O#/O& $%,-% !%" %"+O&/!)!%!&%&!$%!*)!%) %!*$%'"$4-%&'% &''$/ % !*$ % O#U4", % !*$ % '$$4"E % &' % !&#,*"E % !*$ $% 7&+O#!$/%./!%" %'"/+"E%#O;%"C-$O!*%/$'4$,!"&%" % O)"!"E D%!%+)I%U$%)/E#$-%!*)!%"%!*" %,) $;%!*$% /$_#"/$-%&%!*$%" #$ %/$4)!$-%!&%"! %/$H$/U$/)!"& % V&/T%V-%4& $%"! %H" ,$/)4%) O$,!%o%)%'),!%!*)!% V"!*"% &,"$!ID%7$/!)"%O*"4& &O*$/ %)''"/+%!*)!%)/!% $$+ % !&% U$% )% ,& !)! % " % !*$% )!#/$ % &'% )/!;% ) % " % &+$!*"E % !*)! % /$+)" D% 3)/)-&Q",)44I; % )/!% V*$$H$/% &+$!*"E%" %E)"$-;% &+$!*"E%" %)4 &% O/&-#,$-%!*/&#E*%,&+O#!$/%,&-"E%" %$O*$+$/)4%"% 4& !D%i&V$H$/;%!*$%4& %" %'&/%!*$%)/!" !;% $4$,!"E% "! % +)!$/")4 % ) O$,! % )-; % V"!*% !*$ % )-H$! % &'% &$ % +$-"#+ % !&% !*$ % -$!/"+$! % &' % !*$ % &!*$/D% "!$/),!"H$%!$,*&4&E"$ ;%!*$%/&4$%&'%!*$%"!$/),!&/% ($)V*"4$;% !*$ % O#U4", %E)" % ) % O& "U"4"!I% &'% )4!$/ %!*$%$ !)U4" *$-% I !$+%&'%/$4)!"& *"O % ",$% $QO$/"$,"E%U&!*%)4!$/)!"H$ ;% *-%!*" %*)- C !*$%O)/!","O)!"&%&'%!*$%O#U4",%" %&$%&'%!*$%,&/$% &%$QO$/"$,$%U$%)H)"4)U4$a%c>LdD " #$ %&'%!*" %!IO$%&'%)/!D%S*$%,*)/),!$/%&'%!*$%/$C "H$!$-%)/!V&/T%)4!$/ %"! %O/&,$ %&'%,&+"E%"!&% H #.%?-& ,.%C"I$J$'K$'*!,.% " U$"E%$),*%!"+$%"!%" %$),!$-D %@/)A"4%&V)-)I %V$% $$%!*$%U4&&+"E% ()E$!",%/$,&/- %)-%$H$%-"E"!)4%H$/ "& % &'%*),T%4)U %)-%E)/)E$%4)U %)-%+)I%#"H$/ "!"$ % &' % !*$ $ % )/!V&/T % )/$ % O$/" *)U4$D% i&V$H$/;% !*$% *)H$%U$E#%!&%'&,# %&%J4$,!/&",%./!%)-%)%'$V%"% ,&+O#!$/%O/&E/)+%!*)!%/# %!*$+%" %/$,&/-$-%"% 7&+O#!$/ % ./!D% S*$ % '"$4- % &' % )/,*"!$,!#/$>B% )-% *)/-% ,&O"$ % !*)!;% 4"T$ % !*$)!/$ % O4)I % )- % +# ",% -$ "E>p% *)H$%)4 &%U$E#%!&%'&,# %&%!*$ $%"-$) D% ,&/$ % ,) % U$ % -$,&-$-; % O/&H"-$-% &$ % *) % !*$% . %)%$Q)+O4$%&'%!*$%&O$"E%&'%!*$%)/!%'"$4-%!&% T&V4$-E$%&'%!*$%,&+O#!$/%4)E#)E$ %!*$I%V$/$% !*$ $%" #$ %V$%+)I%_#&!$%!*$%$E"$$/%()/,& % V/"!!$D%7& $_#$!4I;%!*$I%,)%U$%/$%$),!$-%)-% 7#AA"&4><% 3*6 % !*$ " % )! % !*$ % 0,*&&4 % &'% /$"!$/O/$!$-; % Y# ! %4"T$% +# ",)4 % ,&/$ %&/% !*$)!/$% 7&++#",)!"&%)-%./! %&'%!*$%G"H$/ "!I%&'%01&% O4)I D%S*$ $%"!$/O/$!)!"& %+)I%)OO$)/%-"''$/$!% 3)#4&;%O/$ $!"E%O/&O& )4 %`'&/%!*$%,/$)!"&%&'%)% $V%+)!*$+)!",)4%+&-$4%'&/%p6%E)+$ ;%" O"/$-%

39 !%& '#!#/$%O#U4",;%Y# !%) %&4-% ,&/$ % &#-%-"''$/$!% )OO/$,")!"E%!*$%U4$-%&'%@/)A"4")%,#4!#/$%V"!*%!*$% &%&#/%" !/#+$! %!&-)ID% )/! ;%V*"4$%'& !$/"E%)%"!$E/)!$-%-$H$4&O+$!D

S*$ % )/!" ! % " % ) % ,/$)!&/ % &' % +$!)O*&/ % I$L$'$%?$ " ,4# !$/"E % *IO&!*$ $ ;% ,&Y$,!#/$ % )- % "!#"!"H$% O$/,$O!"& ;% V"!*&#! % !*$ % "!$!"& % &'% c>d%0)/)%6")+&-D%B??LD%L" #'$&+F(!3"(*'!3"%#')D% -$+& !/)!"E % !*$+;% ) % ,"$!" ! % -&D % i&V$H$/;% %.D%h/&$T$/%)-%(D%h/&$T$/;%$-"!&/ ;%5#&!&*')( V*$% !#-I"E%)/! %*" !&/I; %"!%U$,&+$ %,4$)/%!*)!% %&F&!') ( !1%&";% O)E$ % =KC]?D % G"H$/ "!I% &'% +)I%,&,/$!$%)/!" ! %"!#"!$-%!*$%O& "U"4"!"$ %V$% S&/&!&;%S&/&!&D )/$%4"H"E%)!%!*$%O/$ $!%!"+$D%%S*$%)/!%O/&-#,$-% cBd % l"4m+ % [4# $/D % B??B %M#&!&+FD% J4$,!/&",% !*/&#E*%!*$%O/&E/)++"E%&'%,&+O#!$/%,&-$ %V$% +$-")!"& ;%("$ &!)D )/$%-/$)+"E%&'%!&-)I%V"44%O$/*)O % *)O$%)%$)/% cpd%S)")%[/)E)D%B?>BD%5'#'*)$-&)"D(' "*!(&+( '#!#/$D *$01!"#(6!"$D%%2.(N(/*&"!6(O1#+')D(2( qUH"&# 4I; % !*$ % T"44 % /$ #4!"E % '/&+% P'1!&'+("Q*3'+F"HR;%0O/"E$/CH$/4)E;%j&-&% +)"O#4)!"E % ,&+O#!$/ % 4)E#)E$ % !/) C,&-"'"$-% cNLKD %5$01!"#(0)!!"#( "!&%'$$4"E %)-% $ )!"& %" % !"44%H$/I%4"+"!$-D% '#!D% % ./),I% .+)/)4;% .D% (D% @$44#A&; % % 6m,"&% @/)A"4%+)I%O4)I%)%4$)-"E%/&4$%"%!*" %'"$4-%-#$%!&% 3"E)!)/";%)-%3"$//$%5$ !)I; %$-"!&/ ;%M')%"$'#( !*$%/)O"-"!I%V"!*%V*",*%@/)A"4")% &,"$!I%*) %U$$% 5#%"&#D(1$'('S"+!1#'(%'(#'TUI(O)E$ %><=C>K?;% )U &/U"E % !*$ % ,*)E$ % O/&+O!$-%UI% !*$% (.7CG03;%01&%3)#4&D -$H$4&O+$!%&'%,&+O#!$/%!$,*&4&E"$ D%@/)A"4")% c=d%f)4-$+)/%7&/-$"/&D%>NLKD(2#!"V+&*'D%%./),I% )/!" ! %*)H$%U$$%,*)44$E$-%UI%%!*$&/$!",") %!&% .+)/)4;% .D% (D% @$44#A&; % % 6m,"&% 3"E)!)/";% )-% -$H$4&O%$V%"-$!"!"$ %'&/%)% &,"$!I%V*",*%-$ "/$ % 3"$//$%5$ !)I;%"%&!#;%M')%"$'#(5#%"&#D(1$'( -$H$4&O+$! %)-%),,$ %!&%!$,*&4&E"$ %)-%!*$% 'S"+!1#' ( %' ( #'TUI% O)E$ % >KKC>KN; % (.7CG03;% U$$'"! %!*$I%U/"E%!&%,&!$+O&/)/I%4"'$D%7&+O#!$/% 01&%3)#4&D ./! % +)I% U$,&+$ % &$ % &' % !*$ % ) V$/ % '&/ % !*$ $% cKd%f)4-$+)/%7&/-$"/&D%>NLKD %2#!"(*+*#"!'("(( _#$ ! D% $1+%("Q!"#&#D%%./),I%.+)/)4;%.D%(D%@$44#A&;% . %"!%V) % )"-%U$'&/$;%!*$%G"H$/ "!I%&'% 6m,"&% 3"E)!)/";% )- % 3"$//$ % 5$ !)I; %"%&!#;% @/) "4") % V) % O"&$$/ % " % -$H$4&O"E % /$ $)/,*% M')%"$'#(5#%"&#D(1$'('S"+!1#'(%'(#'TUI%0'F"( O/&Y$,! %"%!*" %)/$)D%! %() !$/%&'%./! %-$E/$$%o% HXYI%(.7CG03;%01&%3)#4&D &''$/$-%'&/%+&/$%!*)%)%-$,)-$%o%V) %!*$%'"/ !%)/!% c]d % f)4-$+)/ % 7&/-$"/&D % >NLKD %Z ( 0#["!( ,&#/ $ % " % @/)A"4 % !&% -" ,# % )- % '&,# % & % !*$ $% *+!#1!&S ( +' ( '#!"< ( % ./),I% .+)/)4;% .D% (D% " #$ % )- % ) % #-$/E/)-#)!$ % U),*$4&/ % -$E/$$ % "% @$44#A&; % % 6m,"&% 3"E)!)/";% )- % 3"$//$ % 5$ !)I;% 7&+O#!$/ % ./! % " % " % O/&,$ % &' % E&H$/+$!)4% "%&!#;%M')%"$'# ( 5#%"&#D ( 1$'( 'S"+!1#' ( %'( )OO/&H)4D% S*" % !IO$ % &' % /$ $)/,*% ) V$/ % !&% !*$% #'TUI%O)E$%]=;%(.7CG03;%01&%3)#4&D ,&++"!+$! % !*)! %E#"-$-% !*$ % $ !)U4" *+$! % &'% @/) "4") % ) % ) % *#U % &' % ,&!$+O&/)/I% H)4#$ ;%

40 c,Ld%./) I%.+)/)4;%.D%(D%@$44#A&;%6m,"&%3"E)!)/"% c>]d%5& )4"-%3",)/-D%B???D %2 "*!&S"(*$01!&+FD% )- % 3"$//$ % 5$ !)ID % >NLKD %M')%"$'# ( 5#%"&#D( (S;%%7)+U/"-E$D 1$'('S"+!1#'(%'(#'TU;%(.7CG03;01&%3)#4&D c>Ld % f)E$/ % @)/Y) % )- % S)") % [/)E)D % B??NN]D %=Q0&8U(7"%&'8:"D% M'+%"#&+F< ( %7)!)4&E#$;%>?@AD ( 2#!"( ( 5'!')F1";%O)E$ %LC>pI% !)#%7#4!#/)4% !"!#!$X% 5$01!'*&+') ( .+!"#'!&S'I;" O)E$ % ]NI" @)T % &'% 01&%3)#4&D @/)A"4%7#4!#/)4%7$!/$;%@/) "4")D c>?d % f)E$/ % @)/Y) % )- % S)") % [/)E)D % B??Nd % ()/,& % [D% 7#AA"&4D % B?>BD %=!'%( M'+%"#&+F< ( %7)!)4&E#$;%>?@AD ( 2#!"( 10"#0!I(3*6%!*$ " ;%G"H$/ "!I%&'%01&%3)#4&;% 5$01!'*&+') ( .+!"#'!&S'I (" O)E$ % ]]I" @)T % &'% 01&%3)#4&D% @/)A"4%7#4!#/)4%7$!/$;%@/) "4")D cB?d % ./4"-& % .D% 0!$O*)D % B?>BD %=+!#" ( ' ( '#!"( c>>d % f)E$/ % @)/Y) % )- % S)") % [/)E)D % B???@AD ( 2#!"( 3)#4&;%01&%3)#4&D 5$01!'*&+') ( .+!"#'!&S'I (" O)E$ % ]LI" @)T % &'% @/)A"4%7#4!#/)4%7$!/$;%@/) "4")D c>Bd % ()/") % j#"A) % [/)E& &D % B??=D %2#!" ( +' ( #"%"( .+!"#+"!D%%()/")%j#"A)%[/)E& &; %"%&!#;%>?@AX% 2#!" ( *$01!'*&+') ( + ( E#'&)I% O)E$ % >?>?;% G"H$/ "!I%&'%@/) "4");%@/) "4")D c>pd % S)") % [/)E)D % B?>BD %2#!" ( *$01!'*&+')D( %& "#"+*&'(6(*+S"#F"+*&'D%%4"))%i$/)-$A% M)/,");%$-"!&/I( \]!&*' ( %" ( )'( -&)F&'( %" ( )( 0&-)"I(O)E$ % L]C>?BI% 3&!"'",") % G"H$/ "-)-% :)H$/"));%@&E&!)D c>NKND %2 ( -#'( '-"#!'D% 3$/ O$,!"H);%01&%3)#4&D c>=d % f)4-$+)/ % 7&/-$"/&D % >NLKD %Z ( 0#["!( *+!#1!&S ( +' ( '#!"< ( % ./),I% .+)/)4;% .D% (D% @$44#A&; % % 6m,"&% 3"E)!)/";% )- % 3"$//$ % 5$ !)I;% "%&!#;%M')%"$'# ( 5#%"&#D ( 1$'( 'S"+!1#' ( %'( #'TUI%O)E$%]=;%(.7CG03;%01&%3)#4&D c>Kd%S)")%[/)E)D%B??=D%_&#!1')&%'%"("(#"')&%'%"D% %6"))%6&+"E#$ ;%$-"!&#I 5#&'8U("(0]!&*'( %&F&!'&;%H&4%LXB;%O)E$ %>p]C><]D%J6G70;%7)Q") % -&%0#4D

41 > S*$%E/&#OX%.4&" "&%./,$4);%@")%($-$"/& ;%i&+$/&%3",&44&;%3)#4&%[&E)});%0#A$!$%l$!#/$44"%)-%S)")% [/)E)D B S*" %!$/+%V) %,&"$-%UI%!*$%!*$&/$!",")%./4"-&%(),*)-&D p [4# $/%"%*" %U&&T%M#&!&+F%# $-%!*$%$QO/$ "&%!$,*",)4%"+)E$ D < ()"4I%M"4U$/!!&%3/)-&;%("4!&%0&E)U$%)-%./!$+" %(&/&"D = 5$E") % 0"4H$"/);% :k4"&% 34)A);% .#E# !&% $ % i)/&4-& % 7)+O& ;% (&" $ % @)#+ !$"; % 0)-/) % 5$I; % 6"))% 6&+"E#$ ;%0"4H"&%v)+U&";%7)/4& %[)-&D K !%" %)%O"!I%!*)!% *$%" %-$)-%)-%,)%&!%),T&V4$-E$%*$/%&V% #,,$ D ] 0$$X *!W !OXW VVVD"!)#,#4!#/)4D&/EDU/W)O4",$Q!$/) W$,",4&O$-")Ä",W"-$QD,'+{ ' # $),!"&Å)/!" !) ÄU"&E/)'")Ç,-ÄH$/U$!$Åp=BN% *!W !OXW VVVD,"U$/,#4!#/)D&/EDU/W!"T"V"T"W!"T"C"-$QDO*O{O)E$Åf)4-$+)/ÉB?7&/-$"/&%

W*!!OXW VVVD+),D# ODU/W+),W!$+O4)!$ WO/&Y$!& W $,#4&QQW+&-#4&pW/#O!#/)W,&/-$"/&WU"&D*!+4%

*!W !OXW VVVD),$/H& D)/!DU/WEHW)/!" !) ÄU/) "4$"/& WU"&ÄV,&/-$"/&DO*O%

*!W !OXW VVVDO"!&/$ ,&D,&+DU/WU/) "4W,&/-$"/&D*!+% .,,$ $-%%.O/"4%>>!*%B?>? L S*$%7&4)U&/)/!$%M/&#O N %\." !*w y)\%'/&+%M/$$TD >? f$%)4 &%*)H$%*)-%"!$/)!"&)4%/$ $)/,*$ %O)/!","O)!"E%"%!*$ $%$Q*"U"!"& %)-%$H$! % #,*%) X%@/")% h)$;%()"-)%f"!*$/ ;%()/E)/"!)%0,*#4!A;%q4"H$/%M/)#;%5)k4%e"|&%@$/)4;%S)I)%6)*+ %)-%7*)!)4% 6#3&!D >> q'%,&#/ $%!*$%$Q*"U"!"&%E#"-$4"$ %)/$%),,$O!$-%UI%!*$%)/!" ! %U$'&/$*)-%V*&%+# !%)E/$$%V"!*%!*$ $% )OO/&),*%)4!*&#E*%V$%),,$O!%!*$"/%&O""& %)-% #EE$ !"& %V*$%!*$I%)/$%O/$ $!%)!%!*$%$Q*"U"!"&% +&!)E$;%V*",*%V$% !/&E4I%!/I%!&%-&%"%)% $4'%&/E)"A$-%V)ID%0",$%&#/%U#-E$! %# #)44I%)/$%H$/I%4&V% &4I%V"!*%!*$%)/!" ! %,&44)U&/)!"&%V$%*)H$%U$$%)U4$%!&%+&#!%!*$ $%$Q*"U"!"& D >B [&/%$Q)+O4$%!*$%@/)A"4")%)/,*"!$,!%M"4'/),&%.4H$ %" %-$H$4&O"E%O/&!&!IO$ %&'%+&-$4 %'&/%+&H)U4$% )/,*"!$,!#/$%# "E%./-#"&%U&)/- D >p S*$%@/)A"4")%-$ "E$/%./4"-&%0!$O*)%)4 &%-$'$-$-%)%!*$ " %/$4)!$-%!&% "+"4)/%" #$ %cB>dD >< ()/,& %7#AA"&4%" %)%@/)A"4")%V""E%E)+$%O/&E/)++$/%)-%" %!*$%6"/$,!&/%&'%!)#%4)U%)!%!)#%7#4!#/)4% !"!#!$D >= %*!W !OXW VVVD O/"E$/4"TD,&+W,&!$!W-L#B>KpL>p<#Lp

42 The Post Digital Art is made of Paper, Cardboard and ABS1

Filipe Pais FEUP – Rua Dr. Roberto Frias, Porto, Portugal DIIP – ENSAD, Rue d’Ulm, Paris, France [email protected]

Abstract The author points out that “it is no longer the This article describes a tactic for media and digital obscenity of the hidden, the reprised, the obscure, but that of the visible, the all-too-visible, the more- art which is based on the displacement of native digital concepts to physical reality. Such media visible-than-visible; it is the obscenity of that which displacement aims to generate fissures rather than no longer contains a secret and is entirely soluble in information and communication.” [2] promote continuity and transparency by contradicting the process of media remediation, Our perspective in line with the famous Marshal giving the spectator the opportunity to reflect and McLuhan’s “The medium is the message” regards understand the mechanisms and effects produced by the medium not only as a vessel for content and message transmission, but rather as a powerful, the complex media web surrounding us. The concept of media displacement was applied and consequent actor, which is able to transform the way tested during an artist residency the results of which we communicate, live in society and define ourselves [9]. support the pertinence and potential of such tactic. The long History of art tells us that technology and Keywords media have been the subject of artists’ inspiration, Displacement; Aesthetics; Remediation; Digital Art. research and declarations for centuries, although this practice seems to be of increasing importance in a 1. Introduction society highly saturated by information and under Media and technology as extensions of our body are media influence. found everywhere and range from eyeglasses, to The article aims to analyze how contemporary wheels of a car or mobile phones and pocket tablets artistic practices can expose media functioning [10]. Lately electronic and digital media appear modes rather than making them transparent, and infiltrated in all manifestations of human life. We how artists can make their audience critically aware are surrounded by thousands of invisible waves and of their relationship with media and technology by by all kind of gizmos and appliances that share our using a tactic of Media Displacement. space and time, connecting us with the others. The observation of recent manifestations in the field Behind the most basic device there is a design of media art, evidences a displacement movement of strategy to make it appear as intuitive as possible to digital native concepts to the physical world, the end user. As a consequence of such contradicting the flow of Remediation. Despite the transparency, the user is immersed in a complex increasing number of observed objects, such tactic digital maelstrom of information, data production has been already observed in artistic practices, and consumption which makes critical awareness of which are not digital based. media use difficult to attain. Jean Baudrillard In a digital paradigm, the representation of observes that the society of spectacle and the drama metaphors such as “window”, “friend” or “recycle of alienation have been substituted by the ecstasy of bin” are certainly familiar to most of us. However, communication, upholding the existence of “a what happens if we displace concepts like “pause”, pornography of information, communication, “undo”, “pixel”, “ftp” or “hyperlink” to a non- circuits and networks.” [2] digital/electronic paradigm? Is this displacement capable of highlighting incompatibilities between these paradigms? Is it meaningful to us as human beings, constantly mediated and surrounded by media? ______1ABS: Acrylonitrile butadiene styrene is a thermoplastic commonly used in 3D printing. 43

Can this tactic render visible the workings of media, A problem seems to arise when art practice uses the breaking the spell of transparency and making us same kind of logic, hiding the medium to provide a aware of how media shapes our perception? frameless space of immediacy and illusion. In this In order to gain insight on the tactic of Media moment of media sovereignty, the artist needs to Displacement, the first part of the text analyses the understand and make visible the workings of media. strategies and efforts of media design to render the They should thus first try to understand how media medium transparent, reviewing the theory of design is thought to be efficient, clear and Remediation by Jay Bolter and Richard Grusin. The immersive. second part of the text, analyzing the work of Aram This takes us to the theory of Remediation which, Bartholl, Christopher Baker, Jens Wunderling and according to David Bolter and Richard Grusin, is a Hiroo Iawata, examines some examples, which fit process that characterizes not only new media into the tactic of Media Displacement. design, but older media as well [3]. Finally, and as this tactic was tested during a multidisciplinary artist residency constituted by 2.1 The double logic of Remediation: Immediacy artists, designers, engineers and architects the last and Hypermediacy section of the text describes this experience and results. David Bolter and Richard Grusin argue that contemporary culture seeks to multiply media and to erase the sense of mediation in the same process [3]. 2. Strategies for transparency Remediation is not regarded as a phenomenon The desire for unmediated interaction and characteristic of digital media but rather as a process experience seems to be the engine for thousands of that has already been observed in previous mediums designers, engineers, ergonomists and an endless like painting, photography, video or virtual reality. array of other actors that work everyday to minimize The authors claim that a medium cannot be the presence of the medium during the users’ considered in isolation since it is always presented interaction. This research follows the rules of a as a refashioned, improved version of a previous one consumer society in search for empowerment in [3]. Media remediation works depending on two faster, smaller, fluid and eye-candy technological logics: immediacy and hypermediacy [3]. extensions. The process of miniaturization and The desire for “live” or real-time, without the media dissemination has infiltrated all ramifications sensation of being mediated demands a design of the personal, social and political, becoming following a logic of immediacy. In order to create ultimately inseparable. more consistent images and interfaces, designers At least two types of immersion are at the origin of and creators assemble different types of media to media transparency. The first type derives from the enhance the sensation of immediacy, using a logic pervasive nature of media and describes our actual of hypermediacy [3]. situation and relation to media. We are immersed by As we demonstrate below, both logics are found in media and have a dependence relation with it. The the use of different media such as video games, character of this dependency is rendered visible cinema, photography, web, virtual reality, etc. when, for instance, we loose our mobile phone or 2.2 Immediacy when the Internet connection is “down”. Such situations are normally causes for frustration and According to the authors, immediacy is already rage rather than moments for detached reflection. detected with the advent of linear perspective, The other type of immersion is related to the former trompe l’oeil, the Camera Obscura and with but it is more specific to the moment of interaction photography, since the latter follows the rules of with the interface itself. While using a certain linear perspective, achieving transparency through medium, the degree of immersion seems to be automatic reproduction [3]. inversely proportional to the degree of awareness of Bolter and Grusin use as contemporary examples for that medium and the more we experience it, the immediacy live TV shows transmitting scenes of more transparent it becomes. The medium seems to war, high-speed persecutions or the cockpit of an F1 appear when we experience some kind of error, bug, car that give the feeling of almost being there. The glitch or latency; otherwise we remain submerged logic of immediacy aims to give to the under its spell. Heidegger calls this moment being spectator/user an experience that is “mediumless” or present-at-hand, observing that we normally use a “interfaceless.” [3] tool ready-at-hand, without reflecting on its The already classic Graphic User Interface was structure or mechanics, although if a hammer breaks built following a design of immediacy. When we while using it, it becomes present-at-hand [7]. observe popular metaphors like trashcan, window, 44 file, folder, desktop or mailbox, we realize that these symbols which “naturally” infiltrate our daily were concepts directly imported from our physical routines but in the quick overlapping and world so we immediately understand how to use combination of paradigms – the digital, physical, them. Today, interactive design research has been online and offline have become interdependent and struggling to produce even more intuitive and difficult to isolate. As the last section will show us, transparent interfaces, mixing digital worlds with the design process and the construction of virtual physical objects that are not just restricted to mouse, and online digital worlds is characterized by a joystick or keyboard but to all kinds of objects that strategy of remediation, which allows for an easier can exist in our physical desks and allow us to adaption to media users. empty the trashcan or to accomplish any other task In this refashioning process, old concepts are in the virtual desktop. imported from previous paradigms but as soon as new functions and characteristics appear, new 2.3 Hypermediacy medium-specific concepts fill-in our routines, Bolter and Grusin characterize the logic of although they are designed to integrate our lives in hypermediacy as a representation of multiple acts natural ways rather than to originate reflection or that become visible. They claim that if there’s a disruption. desire for immediacy it is also obvious the desire for media [3]. They give as examples for hypermedia “Societies have always been shaped more by the the medieval illuminated manuscripts, Renaissance nature of the media by which men communicate altarpieces, Dutch paintings, baroque cabinets or than by the content of communication.” [11] photomontage. In relation to Richard Hamilton’s photomontage the McLuhan and Fiore argue that each medium is an authors observe that: extension of a human faculty or sense. The wheel is for the foot as the clothes are for our skin and the “We become hyperconscious of the medium in electric circuitry is for our nervous system. For them photomontage, precisely because conventional each of these extensions affect the way we act and photography is a medium with such loud historical perceive the world, leaving “no part of us claims to transparency.” [3] untouched, unaffected, unaltered”. The medium is a pervasive massage that “works us over completely” If immediacy is supposed to hide the medium and and in order to understand social and cultural hypermediacy to render it visible then there seems to change one needs to understand and to be aware be a paradoxical relationship. Bolter and Grusin about the media functioning modes and effects [11]. explain this contradiction using the Phenakistoscope Today, in almost every step and action we do, we as an illustration. While a logic of immediacy is are surrounded and mediated by all the kind of found in the moving images that seek for a realistic technological prosthesis that, as McLuhan conveys effect, we go out of the image to contemplate the should be inspected by artists and theoreticians [10]. mechanics and the working mechanisms of the On my way to the office I take a picture with my medium [3]. mobile phone, while listening to the last Sonic Hypermediacy is not always used to render the Youth album. Seating on the bus, I still have 10 medium visible but rather it contributes to the minutes before the next stop so I start adjusting the effective workings of immediacy and, despite the photo’s colours and add some effects using a small different types of media in use, the medium app. I then send it by email to my friends, and post it becomes transparent. Bolter and Grusin give as an on Facebook, sharing it with my extended network example of this use the mix between computer of friends around the world. At work I turn on my generated images and 35 mm video images in film computer, type my password, login and I’m production and the saturated media environments of automatically connected to Google, Skype, TV news that use text, split screens displaying video Facebook, Tumblr, Wordpress, YouTube, Vimeo, footage, live video, motion graphics, static computer and Flickr and many other services which I’m even generated images and photographs to provide a forgetting to mention here. I jump from page to consistent an informative environment that keeps page, feed to feed, and in-between blog and chat the spectator immersed [3]. windows, mail manager and text editor, where I try 3. From Digital to Physical: A Tactic for New to focus on writing the article I should deliver Media Art tomorrow. My work ends and I go back home distributing my attention between SMS’s, phone The complexity of the times we live in is revealed calls to my friends and Facebook news. Finally I not only by the amount of new codes, signs and 45 arrive home and I turn on the TV while I’m cooking the field of digital design. As the authors show, a recipe found by Google. The day ends in the bed digital art has been developing a consistent body of with the laptop by my side and the mobile phone reflective works and digital mirrors, which explore under the pillow. My eyes slowly shut in front of a the sensorial relationship between object and short film playing on YouTube. spectator, although this reflection which at first What this brief analysis of my routine shows is not might appear as an uncanny moment, quickly so much the contents but the enormous quantity of becomes a factor that augments the participant’s communication processes we are intertwined with engagement with the object. This kind of reflectivity during a single day. Following a strategy of is then a factor which contributes more to the hypermediacy, media appears as a continuum, and is account of playfulness and sensorial-reflectivity constructed in a logic of compatibility where than to build a critical account in digital art. everything is connected all the time, everywhere. At this point we should inspect the concepts of The out of format or protocol quickly become Endo-aesthetics and Anti-environment as potential obsolete in a system of objects that converges to models for de-automatization and critical reflection homogeneity. in aesthetic experience. When a considerable amount of the world Claudia Giannetti’s endo-aesthetics derives from the population spends the day mediated by concept of endo-physics. The latter considers that communication protocols, and by all kinds of media humans are unable to observe the world from the we need to agree with McLuhan when he says: “the outside since we are living inside it. As follows, in serious artist is the only person able to encounter order to make this observation possible we need to technology with impunity, just because he is an build model-worlds or simulations [8]. The same expert aware of the changes in sense perception” happens with our relation to media. If we take the [10]. As the mariner caught by the whirlpool in model of endo-aesthetics into account, the artist Edgar Alan Poe’s The Descent into the Maelström should create new points of view about media by the artist, the spectator and the media consumer building anti-environments and counter situations, should be able to understand the “action of the temporarily transforming the fixed point of view of whirlpool” or the “electrically-configured whirl” to the spectator into its own [11]. “staved off disaster.” [11] The tactic of Media Displacement is about the constitution of such media anti-environments. The “If we start to examine the general laws of basic principle of this tactic can be found in this perception, we see that as perception becomes observation by McLuhan: habitual, it becomes automatic. Thus, for example, all of our habits retreat into the area of the “ (…) the meeting of two media is a moment of unconsciously automatic; if one remembers the truth and revelation from which new form is sensations of holding a pen or of speaking in a born. For the parallel between two media holds foreign language for the first time and compares us on the frontiers between forms that snap us that with his feeling at performing the action for out of the Narcissus-narcosis. The moment of the the ten thousandth time, he will agree with us.” meeting of media is a moment of freedom and [14] release from the ordinary trance and numbness imposed them on our senses.” [10] Shklovsky claims that actions, as they integrate our routines and habits, become part of our Following the idea of media hybridization described unconsciously automatic [14]. Media is no by McLuhan in the lines above, we argue that the exception to this, immersing the user and leaving displacement of native digital media concepts and them no opportunity for a distanced view, which their implementation in the workings of physical shows his relationship with media. Bolter and world is able to provide a moment of release and Gromala suggest the existence of an aesthetic of critical detachment from the medium, creating a reflectivity, which is according to them a necessary fission and displaying the incompatibilities between counterpart to transparency that “treats the surface both paradigms. Out of gear, the media fails to as a mirror in which the viewer is invited to reflect work, becoming absurd and uncanny. This moment on her relationship to the work of art on the process of disconnection and discontinuity should be a and the various physical and cultural contexts of source for amusement and critical awareness rather production” [4]. However this concern for an than frustration or a disinterested relation between “appropriate rhythm between transparency and spectators and object. In order to better understand reflectivity” seems to consider mostly the practice in this tactic, the following sub-section analyses some

46 artworks that fit to this trend. These works show and despite the immateriality of the latter, it changes how complex and mediated the world has become our perception and our cognition. Observing this and, how the definition of reality itself has been sculpture or visiting the de_dust building in the transformed by media. Today virtual and online physical world eventually give gamers the uncanny worlds are so wired to our physical “reality” that the experience of already knowing that shape or binary opposition between virtual and real has lost architectural space even though it is the first time its sense [12]. In this text we avoid to use the words they are seeing or visiting it. In WOW a public “real” which refer to our unmediated reality or the performance by Bartholl, the artist analyses online world of physical atoms and the word “virtual” that multiplayer role games and other online is related to digital, numeric and online worlds that environments observing that avatars are always mediate our relationships. identified by a username, which is normally located slightly above the character’s head. He displaces 3.1 Displacing Media this concept of avatar identification to the physical The tactic of Media Displacement has been world of human social interaction, by constructing observed more commonly in three types of the same kind of identification using cardboard and artworks. The first type uses non-electrical and non- a very light and transparent structure to hold it over digital media materializing concepts or situations, a person’s head. This social experience displaces a which are normally immaterial and native to a pervasive specificity of online digital environments, digital-online paradigm. In the second type of work, that is familiar but at the same time transparent to the artist normally uses digital media, although us. The uncanny experience created by Bartholl there’s also an on-going materialization process. In highlights not only formal and functional the works of Christopher Baker and Jens incongruities between a physical and digital Wunderling for example, online data becomes paradigm but more importantly asks: “what happens materialized, occupying the physical space in the when a person’s customary anonymity in the public form of paper. Finally, the artist might perform this sphere is obliterated by the principle operating in transposition by using digital devices or apparatus, virtual worlds?” [1] which transform our relation with the physical The work default to public: tweakleak by Jens reality, re-creating a perspective or phenomenon that Wunderling seems to share similar concerns as is characteristic to the digital worlds. An example of WOW. In this networked public installation, inside a this would be a device, which allows us to undo an café in Berlin, we find a printer installed inside a action in the physical world or to pause and freeze monolith which prints twitter messages in small time like in a video game. sticker strips. Each time a tweet is sent in the Aram Bartholl has been working with non-digital surroundings of the coffee, a sticker with the tweet’s media, creating objects and performances that use content is printed in the café and the author is raw materials like paper, cardboard, plastic or other notified over twitter with the message: “Your tweet ordinary low-tech objects. The strength of his work has been taken away” [15]. With this apparatus, resides precisely in the use of such simple materials Wunderling connects two spheres which are that tell so much about the mechanisms of high normally separated and work very differently, first technology and about our relationship with them. questioning the sense of privacy online and in the Bartholl has been mainly interested in the physical world and then creating an awareness of displacement between online world and offline self-exposure [15]. By leaving the digital online worlds. de_dust is one of the world’s most famous world, the tweet enters the physical world that has maps from Counter Strike, an online first-person different mechanisms of information processing and shooter game. Thousands and even millions of the author looses track of his message as anyone can online gamers know this map better than the map of take away the sticker and use it as they want. This their own city or the neighbourhood they live in. So, loss of control seems to highlight the incongruities departing from this idea, Bartholl built an Abs between both paradigms and, instead of solving plastic sculpture, which is a 1:1 scale replica of them the artist uses it as part of the work’s de_dust. This replica is the model for a large-scale statement. By informing the tweet‘s author that his building that people should be able to visit in a near message has been delivered and set “free” into the future as a “monument of cultural game space physical space, the artist is producing a kind of heritage” [1]. With this sculpture, Bartholl reminds distancing effect allowing the user to reflect on the us about the place that technologies and media medium’s functioning mode and on his own use of it occupy in our life, shaping our bodies, memory, [5]. relationships and society. He shows us that today the “real” and “virtual” worlds are deeply intertwined 47

The intense and unstoppable activity of Internet proprioceptive and sensorimotor knowledge of our produces a quantity of information that would be bodies. It transforms one of the more natural, impossible to manage and to archive in a physical familiar and transparent tasks – walking around – paradigm. A big part of the information, which into a new uncanny experience which demands an constitutes this flow, is related to personal data and additional effort, eventually causing a reflection in emotional information from users. This data is our “internal” processes which are normally recorded, archived and in some cases eventually invisible and taken for granted. sold to marketing corporations. Murmur Study, an Normally, the process of interface and digital design installation by Christopher Baker, transforms a body as explained by Bolter and Grusin adopt an inverse of immaterial data, into a huge mass of paper which logic, embedding physical familiar objects that is dumped and printed by 30 thermal printers [6]. In surround us in “new” sophisticated objects and the long lists of paper that progressively form a interfaces that smoothly enter and adapt to our chaotic hank on the ground, one is able to read routines. The works by Bartholl, Baker and “emotional” messages that are continuously taken Wunderling displace digital and online medium- away from twitter status in words like: arghh, meh, specific concepts to the world of physical matter not grrrr, oooo, ewww, and hmph [6]. But Baker doesn’t to make our life easier, faster and simple as on a want us to read all these messages, but rather to design basis, but rather to call our attention to the place us in front of the impossibility to access all workings of media that we have allowed to enter our this data. By doing this he is allowing us to reflect lives without questioning. These displacements then on the medium and on the way we use it, alerting us produce uncanny experiences by transforming for the fact that one can use that information as they familiar concepts into contemplative and opaque want without even being noticed. This non- objects that become freshly new to us. Domenico functional way of displaying information is opposed Quaranta argues that our reaction to these objects to a strategy of remediation design, showing the might vary according to our experiences, works of media. The apparatus brings to light background and relationship with them. They will incongruities between the digital world of twitter however always produce a shock and an and the protocols of information exchange in a non- uncomfortable sensation to us [13]. electrical paradigm, that is not equipped to process, This tactic can be further explored and can provide display and archive data in the same way digital digital and new media arts a strong basis to develop media does. artworks and experiences which go beyond the ludic The third type of displacement is observed in The and sensorial, forming a critic and detached Floating Eye, a work by Hiroo Iawata, in which the spectator who becomes aware of the processes that artist builds an apparatus that allows the users to see media tends to hide. The best way to explore such themselves from above, as if the person was the tactic is to put it in action, test it and see if it can avatar from a video game. Iawata displaces the open new paths for reflection and representation. person’s field of vision by placing a camera in a 4. Testing our Tactic small airship floating a few meters above the head of the participant. Then the captured image is sent to During Guimarães 2012 - European Capital of a small monitor installed inside the helmet carried Culture, the LCD LAB from Porto (Laboratory for by the participant [9]. Digital Creation) invited me to organize with their The experience unfolds by acting in a world where team an artist residency, which applied an unusual the spatial perceptual space appears distorted. Here experimental model of work. The residency took shape, size and distance affect the way we move place during 6 days, from the 12th to the 17th March around and our visual perceptual system is affected of 2012 at the CAAA (Center for Arts and by the wind that we need to take into account when Architecture) in Guimarães. In the first two days moving. The strangeness of this experience is four theoretical sessions were presented and caused by the slightly random character of vision intertwined by discussion and brainstorms with the (originated by the wind) and by the disconnection coordinators and the groups of residents. The rest of between movements and visual field. the days were dedicated to prototyping. Iwata explores the concept of first person view, My contribution to the residency was a theoretical familiar to gamers in first-person games, building an exploration and the conceptual framing of the tactic experience that acts like a mirror for our bodily of Media Displacement, introduced in this article. actions and functions. More than reflecting on the This session analyzed the strategy of Remediation use of the medium, this experience produces by Bolter and Grusin, the visions of media and intermodal conflicts between sensory stimulation, technology by McLuhan and Heidegger and some works which fit in the trend of Media Displacement. 48

Ricardo Lobo, Pedro Ângelo and Sérgio Ferreira “I”, an installation by Pedro Ângelo, Rebecca presented other theoretical sessions which discussed Moradalizadeh and Sónia Ralha remind us that the and analyzed the concept of hyperlink from a physical space we inhabit is imperceptibly changed sociological, philosophical and mathematical and fulfilled by thousands of invisible waves perspective. At the end of three intense days of coming from all the technologies and media that discussion and reflection, the four groups, surround us. By entering the installation space, the constituted by 12 elements with backgrounds in fine visitor becomes an involuntary participant in the arts, design, architecture and engineering, came up soundscape composition, which emerges from the with four ideas which were then prototyped. At the interferences triggered by the visitor’s mobile end of the week, four prototypes were presented: phones. When approaching one of the seven cupper- “Join”, “I”, “AHA! – A Human Approach”, and cables vertically placed between the ceiling and the “3.000.000 Views”. ground, the electromagnetic field of the participant is transformed into a noisy sound mass, which 4.1 Projects Developed grows and becomes chaotic when more visitors enter the space. After the third day, the groups started to discuss and “AHA! A Human Approach”, is an interactive to materialize their concepts, exchanging their installation by Gil Ferreira, Mónica Mendes and knowledge and collaborating in-between them. In Victor Diaz, which explores the mechanisms of the last day, the prototypes were presented in a online research. A fake Google search page presents public session at CAAA. the results in the form of videos, instead of the “Join” is a performance developed by Ana Palma, habitual list of links. In these videos we see a group Diana Combo and Tiago Serra exploring the of three operator-researchers, searching in books, transposition of actions between online and offline tasting, smelling, experimenting and going out in the social contexts. Each person that joins the Facebook nature or discussing in between them to find the best event corresponding to the performance has answer. AHA! turns the transparent process of someone waiting for them in the offline event (the online search into the content of the installation, performance itself), hanging a sign with that breaking our expectations and showing how person’s name. This confrontation aims to introduce automatized we become by technology. In a a critical distance to our use of the medium and to moment marked by optical fibre speed and world our notion of privacy, inclusion and exclusion in the Googlization, this project inverts the logic of transition from online to offline. transparency, objective and immediate, by slowing down the research process in an attempt to reflect on the impact of mediation and on the contemporary search-engine paradigm. “3.000.000 Views” is a public installation and performance developed by Raquel Carteiro, Sandra Araújo and Sérgio Ferreira that uses a popular strategy for online marketing. In different placards spread around the city one can read “3.000.000 Views” in black letters over a white background, with a QR code just under the text. By reading the QR code with the mobile phone the passer-by becomes an involuntary participant of this installation and performance. They see an image of someone looking through a wall somewhere and then a map indicating this location. If everything goes as planned, the participant will go up to the gallery where the hole is located and when looking through the hole, they will see a man seated in a desk, which stands up when he realizes he’s being watched. Then he draws a line on the wall with a piece of chalk, which means another “view.”

Figure 1 | Poster for performance “Join”

5. Conclusion References [1] BARTHOLL, A. (2012). The Speed Book. Gestalten. The Media Displacement tactic used works during [2] BAUDRILLARD, J. (1998). The Ecstasy of the LCD Artist Residency presents a potential in the Communication. Semiotext p. 11-27. creation of digital media and interactive artworks, which needs further attention. Our hope is to see [3] BOLTER, J. D., GRUSIN, R. (2000). Remediation: more digital and non-digital artworks that are able to Understanding New Media. MIT Press. create critical insight and render media visible rather [4] BOLTER, J., GROMALA, D. (2004). Transparency than transparent. Media Displacement contradicts and Reflectivity: Digital Art and the Aesthetics of the movement of media remediation, for the sake of Interface Design. Ed Aesthetic Computing. MIT Press. a critical look beneath the transparent layer of the interface, grasping the hidden underworld of code [5] BRECHT, B. (1964). Brecht on Theatre: The and algorithms. Such displacement breaks Development of an Aesthetic. Ed. and trans. John Willett. London: Methuen. functionality and displays incompatibility, providing a moment of uncanny apperception. Finally the post [6] BAKER, C. 2009. Retrieved May, 2012, from digital art could be made of any kind of material or http://christopherbaker.net/projects/murmur-study/ use any media – paper, plastic, biological tissues or [7] HEIDEGGER, M. (1978). Being and Time, Wiley- electronics as long as it avoids their use for the sake Blackwell, London. of the new and the spectacular but rather face it [8] GIANNETTI, C. 2004. The Spectator as Interactor: critically. Myths and Perspectives of Interaction. Lecture in CGAC Santiago de Compostela. Acknowledgements [9] KUSHARA, M. 2007. Device Art: A New Approach in Understanding Japanese Contemporary in Media The author wishes to acknowledge the assistance Art Histories edited by Olivier Grau. MIT PRESS. and support of the advisor Patrícia Gouveia and all the discussions with Samuel Bianchini and [10] MCLUHAN, M. 2007. Understanding Media. Emanuele Quinz from DIIP – ENSAD – Paris; LCD Routeledge Classics. Lab Guimarães and all the elements of the LCD [11] MCLUHAN, M., FIORE, Q. 2008. The Medium is Artist Residency 2012 and Vera Almeida. This the Massage. Penguin Books research is supported by FCT (Fundação para a [12] QUARANTA, D. 2011. Do We Really Need the Ciência e Tecnologia) with a PhD scholarship Virtual in Art? The Australian Journal of Virtual Art. (SFRH / BD / 47277 / 2008). Issue 1: September 2011 – ISSN 1839-5481. [13] QUARANTA, D. 2012. The Ethics of Dust from The Speed Book. Gestalten.

[14] SHKLOVSKY, V. 1965. Art as Technique. Russian Formalist Criticism: Four Essays. Ed. Lee T. Lemon and Marion J. Reiss. Lincoln: U of Nebraska P, 1965.

[15] WUNDERLING, J. 2009. Retrieved May, 2012, from http://jenswunderling.com/works/default-to- public/.

ON THE SEPARATION BETWEEN THE we will attempt to establish a specu- ESTHETIC AND THE FUNCTIONAL lative vision of how digitalization, And how the digital realm will steal and thus the transformation of all form things into information, will inhabit the very shape of objects and affect Abstract our understanding of esthetics. Looking at the impact of cyberspace over society and culture, we can ob- The new technosocial paradigm serve that form tends to become the Only a quick glance at the cityscape visual vessel of the virtual self. and its people walking eye-on-palm There is a foreseeable possibility for through the streets is enough to tell our society has been shaken by pro- esthetics to dematerialize in the de- st signed environment. This study aims at gress in this freshly born 21 centu- asserting this hypothesis and antici- ry. The frenzy for communication that pates on a new approach to the formal- appeared in the trail of two Industri- ization of esthetics. al Revolutions and globalization led to the constitution of an immense pool Keywords: self-representation, esthet- of data. Data, initially considered as ics, cyberspace. the only imprint left over by communications, now weaves a yet uncertain Introduction and vague territory we call cyber- Today's society is admittedly charac- space. From a simple document exchange terized by the fast-growing importance system developed to fulfill post World of the digital realm. The ever-more- War II dreams of hypertext [Vannevar social web has brought radical changes Bush, 1945], Internet, last and most to how we share culture and conse- prodigal child of ultra-communication, quently, to our perception of self. has become the best candidate for navigating this space. Design, essentially characterized as the productization of culture, has yet Now, this big data pool is an ever- to transcend postmodernism. As carou- growing one: YouTube recently an- sels of new digital and visual memes nounced an average of 72 hours of vid- emerge and propagate constantly, do eo uploaded every minute to their new objects need to exist when solely servers, while Google's index has conceived around storytelling? What reached 100 million gigabytes and four will be the esthetic manifestation of billion pictures are uploaded to Face- our material culture if successfully book every month. A study conducted by brought into adequacy with the digi- IDC foresees that we'll go from a tal? Will new objects have to be made global digital data footprint of 800 from scratch to accommodate new memes? billion gigabytes in 2009 to 35 tril- Are objects updatable? And more im- lion gigabytes in 2020. portantly: is functionalism still screwed tight to esthetics or are we Cyberspace has become the new El Dora- heading towards a complete separation do. It is a new territory to explore of the semantic and useful? and conquer, where pioneers from all over the world come to plant their After a brief look-back at the boom of flags. Companies have more or less communication and its latter economic, successfully adapted their marketing social and political implications, we techniques to the web, trying to pursue the capitalist dream in a highly will see how objects semantically − or unstable and virtual market. Tradi- rather, esthetically − adapt in turn tional media are also here to claim a to these observed phenomena. Finally, slice of the pie by attempting to save

51 their dying industries4. Comes then so- Such radical change is clearly the ob- cial networking, demolishing borders servable general characteristic of an by turning cyberspace into a distrib- abundant society. Despite it not being uted network of individuals (most re- an abundance of goods, it is abundant cent examples being Facebook in 2004 in bits of information. And as infor- or Twitter in 2006). On the other mation gains value, one could say our hand, wikis and blogs proliferate and entire sense of value is being digit- federate as the first steps towards ized. Past the digitalization process, collective intelligence of future cre- social, political and economic pat- ative conversations [Pierre Lévy]. Cy- terns emerge. In order to address the berspace has turned out to be so rich web’s foreseeable impact on form, let – too rich – in data that it now seems us first outline three of such rele- to require an alphabetization to col- vant patterns. lective intelligence [Pierre Lévy] or the development of a semantic web The first pattern is that tomorrow’s where every single piece of data would society seeks transparency, as it is have an http address [Tim Berners- necessary for information to be acces- Lee, 2009]. sible and useable. The way we look at politics, for instance, changed with Have we already shifted to a digital the birth of cyberspace; the Arab economy? How much have we converted spring, Wikileaks, the emergence of atoms into bits6? How can we tell? Anonymous or the Occupy movement are Well, if we believe Bruce Sterling's strong occurrences of our craving for interpretation, we know there has been a quick and free access to facts. a revolution in technoculture when States that would fight hard for in- that technoculture cannot voluntarily tegrity believed to come with the return to the previous technocultural opacity of a state-nation are now em- condition [Sterling, 2004]. As the in- bracing transparency. Even the Obama vention of writing has made us think administration is now glorifying ourselves as parts of a history in- transparency as a key principle for stead of existing as phenomenal be- efficient governance, while opacity is ings, and as telecommunication has becoming scarce and highly coveted – made us think ourselves globally in- mostly because the governments' claim stead of locally, the creation of a for transparency often disguises per- cyberspace is making us think of our- sonal data gathering and sophisticated selves as data-carrying agents in a means of surveillance. Traditional me- distributed network abundant in data. dia, newspapers, magazines, televi- We just can't go back, mainly because sion, are all wiping their ways to- an ever-growing part of our society is wards better transparency, for it is now depending on quick data consump- an epic win. Data gets to be produced, tion and exchange. enhanced and filtered by users themselves. Moreover, users generate data without necessarily knowing it. Compa- 4 The impact of the information abundance means nies like to have a hand on society’s that those goods and services previously based pulse. So, transparency is actually a upon information scarcity and its high costs of win-win. As such, not many people are reproduction, such as journalism, films and mu- left to oppose the process of making sic are severely challenged in finding viable methods of generating revenues, writes Aaron everything public. Peters on his web-published article The crisis of value in the network society. Another remarkable pattern lies in dy- 6 Expression borrowed from Nicholas Negroponte, namism. Everything happens in real- who talks about the implication of this conversion of atoms (material goods) into bits (digi- time. From live surveys on the web to tal goods) in his book Being Digital (1995), geo-localization on connected mobile

52 devices, everything happening here and It appears often on the web that the now must be leaving an imprint onto frequent, loud and unconditional the cyberspace – Facebook's timeline statement of our smallest action in- is a result of bringing the semantic fuses us with uniqueness. "A burnt im- web into social networking – and user- age of Ray Liotta has appeared on my tracking is a consequence of such a tortilla" tweeted on the 23rd of May10. process. Only, it becomes less of an issue for as long as we adhere to the Now, as geographic distance shrinks cult of transparency. before faster communication, there is a nuance to how the end-user manifests A dynamic society is capable of emit- him/herself in data. Society is the ting signs constantly while ubiqui- victim of an even more violent intrin- tously projecting them at any end of sic spectacular separation [Guy the network. One is not only able to Debord, 1967] due to the complete connect and correlate already availa- abolishment of distances, where the ble information, but also to communi- individual has become the spectacle cate by stigmergy9 over the semantic and data, its currency11. What matters web. All signs emitted by humanity is not what information you bring onto tend to join the universal visible the cyberspace – for abundance makes sphere of the cyberspace [Pierre Lévy, it unnoticeable – but the way you or- 2002]: everything we do, think, pro- ganize it, catalogue it in a coherent, duce, is a stone thrown into the data logical, rational and self-centered pool. It has an impact, and it shakes system that is a reflection of your- the whole with varying intensities. self: your virtual identity. That phe- Dynamism occurs when we can address nomenon is also a result of infor- such changes in real-time. mation abundance, for it creates an economy in which the only available Our third relevant pattern here is measure of competitive success is rep- egocentrism. It occurs as a conse- utation among one's peers [Eric S. quence of the first two patterns in Raymond, 1999]. One could say we’ve that the self wants to be discovered, already acknowledged the virtual self observed and followed. One gladly as a prime and legitimate focus for turns into information too, and does marketing, in a spectacular wave of it quickly, being both transparent and which you are the star. dynamic. Foursquare allows me to tell others where I am , Pinterest is for Living in a material world what I like… Transparency may be As we’ve now observed how intangible, linked to a craving for honesty, as we information-related matters have sub- saw, or a desire for openness follow- mitted to transparency, dynamism and ing an even greater desire to optimize egocentrism, what about physical mani- global communication. But next to a festations of information? What about rising sense of individuality, isn't objects and their role as cultural ar- it just as much of a personal and ex- tifacts? What importance do we give to istential need to be seen as unique? our daily tools and decorative knick- knacks? What do they mean? And more importantly, if they are emitting 9 Through his academic work, Prof. Pierre Lévy signs, will those signs also enter the from the University of Ottawa uses stigmergy, as an example for the way people navigate and communicate through the cyberspace. Stigmergy is a principle, common for insects like ants or ter- 10 By @cedricbixler_zavala, permalink: mites, which consists on altering the environ- https://twitter.com/cedricbixler_/status/2051941 ment as a mean of communication, in order to 63820572673 increase the performance of the next action tak- 11 The online service Pay with a Tweet is our en by the colony. best example for that phenomenon.

53 cyberspace? The very shape of every- Looking closer, the industry has thing has to be re-thought in order to adopted the model of an information match the newly established semantic economy, where the semantic value of territory. things serves representations of cultural archetypes. Objects are the scene As we move away from the modernists’ of a golden age of branding and demo- “form follows function”, beyond the graphic targeting; a world full of do- undividable essence of an object’s not-sit-chairs exhibited in galleries necessary form serving its purpose, and museums; a world full of designer there is a part of its shape that products, getting further and further tends to vary with time, trends, po- away from the original Idea of the ob- litical context, social position, sta- ject. Such objects are reduced to me- tus and self-representation – as it is dia for storytelling. not quite a reflection of yourself but an imaginary, spectacular representa- Contemporary objects are just not tion of yourself in the social game. matching the new technosocial environ- That extra essence, that little je-ne- ment we were describing earlier. They sais-quoi, is an object's semantic are de-standardized and still mostly value. We seek to convey something brand-identified storytelling plat- about ourselves (or our products or forms focusing on the author rather places) not only through verbal decla- than the user. They are also static, rations but through look and feel, for they are frozen in time and space creating, in the process, new connec- by the material impossibility of dy- tions between surface and substance, namic manufacture. form and meaning. We want to show as well as tell, and the increasing vari- Yet this is changing, and in order to ety of aesthetic options permits more understand what's going on we will complex or differentiated meanings, a take a look at two contemporary phe- more specific match between outward nomena. form and inward identity [Virginia Postrel, 2003]. It is what we identify First is the death of postmodernism- to, what shapes our cultures. Decora- driven economics and design. Postmod- tive objects, for instance, are prod- ernism, like modernism and romanticism ucts which existence is legitimized before it, fetishized the author, even only by their semantic value. So do when the author chose to indict or tools or instruments, though purely pretended to abolish him or herself. conceived around functionality. But the culture we have now fetishizes the recipient […] to the degree that Objects are technosocial entities, and they become a partial or whole author their very shape is driven not only by of it [Alan Kirby, 2006]. This relates the Idea of an object but also by the to the egocentric society described semantic value instilled by mankind, earlier; our culture is dying for for the producing man participates means to adapt to the customer, and himself […] to the way a product comes the end-users we were, are now slowly to being [Heidegger, 1949], hence the evolving into agents in complete cult of storytelling driving our con- charge of insufflating meaning to ob- temporary society since the fulfilling jects. of the postmodernist vision. As basic function-serving tools and objects On an industrial level, there is only have become abundant, it is their se- one attempt to grant users with lim- mantic value that has now become mar- ited esthetic choices while keeping ketable, as a reflection of the social mass-production intact, namely: cus- spectacle.

54 tomization. However, it doesn't really to a material existence. So where did allow for more than the customization the crazy different shapes go? Well, of a few details, or rather, it they have just shifted into the cyber- doesn’t customize enough to grant the space. The smartphone is a non-object: users with full authorship. Products a semantic platform limited to its un- never truly come as a blank canvas for dividable and solely functional shape. the user to brand. Customization has On the other hand, Apps inherit the failed, as it ‘feels fake’12. look of physical interfaces while the physical platform tends to disappear. This brings us to a second point: Here lies the first step in the sepa- smartphones, or, the epitome of user- ration of the semantic value of an ob- enabling objects. If you take a look ject and its function. at our mobile phones from 10 years ago, you will see a large amount of How can tomorrow's objects reflect any colorful and differently shaped phones more the technosocial environment? How with, although limited, possibilities can an object become dynamic, trans- of customization. The phone now became parent and egocentric enough for it to a connected mobile device capable of take place as the new cultural arti- emitting and receiving signs in real- fact? Our last part proposes prospec- time, to and from the cyberspace, and tive guidelines for shaping future ob- allowing infinite means of virtual jects. customization ranging from wallpapers to organizing and selecting Apps. They Draft for contextual dynamic and se- provoked the new ritual of forming mantic displays one’s personal approach to navigating Concerning transparency, the now popu- data. Since the smartphone era, a very lar adaptation method of products to particular shift can be observed in this phenomenon seems to be permanent these objects’ design: they tend to connectivity to the Internet. Connect- re-standardize. New means of customi- ed objects would exist both physically zation and organization have gained and on the web, thus expressing seman- relevance in user interfaces rather tic value in both realms. The recently than in physical objects. As possibil- overfunded Pebble Watch, or the auton- ities for furnishing smartphones mul- omous GPS-based cars15 are other exam- tiply across the cyberspace, their ples of this aim for connectivity. One shape has returned to a basic func- by its being another platform for per- tion-serving one. They have turned in- sonal data to be manipulated, and the to thin black boxes and barely differ other for its being literally driven from a brand to another. As if they by data. Looking from afar, both are were intended to disappear or remain mere vehicles for personal data. stand-alone screens, if only designers were able to get past the necessary Tomorrow’s object will have to become battery, SIM card holder, microphone, egocentric, which is probably the eas- webcam and speakers, binding the phone iest thing to imagine, as everything in contemporary culture is already 12 According to Joseph Pine, author of Mass Cus- centered on one’s personality, or ra- tomization (1992), it seems that contemporary ther one’s profile. It bears no doubt consumers only feel happy when spending their that future objects will have to cele- time and money satisfying their desire for au- thenticity. A desire that companies in charge of the manufacturing of goods are incpable to satisfy, being only able to render it with more and 15 After Nevada in March 2012, a bill just less efficiency. Manufactured products cannot be passed California's State Senate authorizing the truly authentic - unless they allow the user to use of driverless cars on the roads, thus let- be in complete charge of the customization pro- ting us believe that research in that area might cess. intensify in the near future.

55 brate users even more. Designers bound to material constraints and should now be more preoccupied by de- static means of production, which fa- signing the designer, as storytelling tally lead to static objects. is being re-appropriated by end-users becoming authors of their own esthetic But does shape need to be tangible in rules and therefore, of their own se- order to be visible? This would imply mantic expression. There is a necessi- that what doesn't serve a physical ty for future objects to become function, meaning what is no longer screen-objects that can display, modi- bound to the material world, might fy and share data in a way where the just be destined to complete demateri- author – the designer or the brand – alization – leading us to art. A lot would become completely transparent, of web-galleries and art blogs now ex- and the end-user would be glorified. hibit artwork, which holds no other existence than a digital one. If art Lastly, objects have to become dynam- is no longer required to manifest ic. They must be able to react to the physically, does it question its being stigmergic shifts on the cyberspace, a piece? What will the world look like which happen in real-time, and formal- once esthetics become an entirely dig- ize in an object’s visual qualities. ital layer? As treatment of information becomes crucial, and as the way one relates to Technologies like contextual augmented data constitutes one’s reputation in reality aim to become a solution for the economy of information, it seems the display of ever-changing semantic extremely important that an object’s shapes, contextually attached to the shape adapts to contexts, situations physical and undividable form of the and people in real time. One could object – the functional object17. draw inspiration from infographics which are digital but dynamic repre- All it takes for augmented reality sentations of data. Let’s imagine ob- (AR) to popularize is an easy and jects as physical infographics, of quick interaction between the user and which every esthetic characteristic is this new augmented layer of meaning designed to be variable and reflecting via user-friendly interfaces, as well of its data. The designer’s role ends as appropriate methods of visualizing up being bound to the making of a it. AR has been brought to life in the platform, not unlike mobile tablets, already popular smartphones, webcams, allowing such changes on the user- projectors, and soon to be goggles, side. while bionic lenses are on their way.

Looking at fashion items, or objects Conclusions in general that are intrinsically con- The object’s physical shape won’t dis- nected with emitting personal signs in appear, for as long as we live physi- society, our speculative object could cally and require instruments to feed, be a hat, for instance, whose shape is influenced by how we feel and what we believe that one day abundance of goods might want to reflect at a given moment. It become possible through a re-appropriation of would be like one’s appearance is di- the means of production by the consumers. 17 rectly submitted to an online profile. Augmented Reality has been seen by many artists, speculators or futurists as one of the And although freeform fabrication is most promising technologies, and now is the sub- at hand16, an object’s physical form is ject of research all over the world. We can, for the example, quote Microsoft's Future Visions video series or Google's proposal for Augmented 16 The Fablabs, the RepRap project, CandyFab, Reality glasses. The latter very recently an- along with an ever-evolving research on printa- nounced (May 31st 2012) these glasses to go on ble food, organs and organic tissues lead us to sale next year.

56 move and communicate. But its semantic Open Publication License and viewable value, encountered in its esthetics, online at: is bound to join the cyberspace, which http://catb.org/~esr/writings/homestea is already a treasured test-ground for ding/homesteading/ new esthetic possibilities. Virginia Postrel, 2003. In The Sub- For that matter it appears clear that stance of Style. the correlation to the meta-data of objects will have to occur over a con- Martin Heidegger, 1949. In Der Ur- textual display, a second layer of sprung des Kunstwerkes (The Origin of shape to be defined by the user, for the Work of Art). the user; a dynamic display method that would support the necessary con- Alan Kirby, 2006. In his article The nectivity to our complex web of seman- death of postmodernism and beyond pub- tic value. This would get to express lished in Philosophy Magazine and esthetics independently from their ma- viewable online at: terial, functional counterpart. http://www.philosophynow.org/issues/58 /The_Death_of_Postmodernism_And_Beyond References Vannevar Bush, 1945. In As We May Guy Debord, 1967. In La Société du Think, Bush emits the idea of hyper- Spectacle (The Society of the Specta- text, through the proposition of a cle) provision whereby any item may be caused at will to select immediately Web References and automatically another. Tim-Berners Lee and chronology of the invention of the World Wide Web, Pierre Lévy. In Vers une science de http://designmuseum.org/design/tim- l'intelligence collective (Towards a berners-lee science of collective intelligence). 2010 and La sphère publique du 21ème MITSloan article by Renee Boucher Fer- siècle (The public sphere of the 21st guson about 'big data' on the Web, century), 2011. http://sloanreview.mit.edu/improvisati ons/2012/05/22/big-data-service-from- Tim Berners-Lee, 2009. In a TED talk the- about the future of Internet, Tim cloud/?utm_source=feedburner&utm_mediu Berners-Lee insists on the necessity m=feed&utm_campaign=Feed%3A+mitsmr+%28 of a semantic web. Talk viewable MIT+Sloan+Management+Review%29#.T79JCb online at: 9iPS5 http://www.ted.com/talks/tim_berners_l ee_on_the_next_web.html Article on Wired about the global data digital footprint, by Dylan Tweney, Bruce Sterling, 2004. In Shaping http://www.wired.com/gadgetlab/2010/04 Things, pages 9-11. The term technoso- /here-comes-the-zettabyte-age/all/1 cial, used many times through this paper, is also borrowed from the same Article by Evgeny Morozov about data book. collection and transparency in the cyberspace, Pierre Lévy, 2002. In Cyberdémocratie http://evgenymorozov.tumblr.com/post/2 (Cyberdemocracy). 0347146711/my-piece-on-the-history-of- facial-recognition Eric S. Raymond, 1998. In Homesteading the Noosphere, essay on creative com- mons and the hacker economy, under

57 Aaron Peters' article, The crisis of The presentation of the RepRap project value in the network society, by its founder Adryan Bowyer, http://partitaimaginaria.tumblr.com/po http://vimeo.com/5202148 st/21614104709/the-crisis-of-value-in- The CandyFab wiki, the-network-society-the http://wiki.candyfab.org/

Barack Obama's Memorandum on Transpar- A NewScientist article about living ency and Open Government, tissue created by ink-jet printing, http://www.whitehouse.gov/the_press_of http://www.newscientist.com/article/dn fice/TransparencyandOpenGovernment/ 3292

Jamais Cascio's article about opacity A CNN money article presenting a 3D in the cyberspace, printer capable of making edible food, http://www.openthefuture.com/2012/04/o http://money.cnn.com/2011/01/24/techno paque_projections.html logy/3D_food_printer/index.htm

Various websites or social networks References on augmented reality: evoked in this paper: Microsoft's future visions video se- Facebook, www.facebook.com ries, Twitter, www.twitter.com http://www.microsoft.com/office/vision Pinterest, www.pinterest.com / Foursquare, www.foursquare.com YouTube, www.youtube.com An article announcing the commerciali- zation of Google's augmented reality The Pay with a Tweet website, glasses, http://www.tech.sc/google- http://www.paywithatweet.com/ project-glass/

A very relevant image on the re- An article by Jamais Cascio on the standardization of mobile phones, possibilities of augmented reality in http://appadvice.com/appnn/2012/02/cle future fashion, ver-image-highlights-mobile-hardware- http://www.fastcompany.com/1568276/aug before-after-iphone mented-fashion-reality

The Pebble Watch Kickstarter page, An article on possible future applica- http://www.kickstarter.com/projects/ tions of augmented reality, 597507018/pebble-e-paper-watch-for- http://www.howstuffworks.com/augmented iphone-and-android -reality.htm

Article about laws on driverless cars The TED presentation for a primitive in California and Nevada, augmented reality device, http://www.latimes.com/business/money/ http://www.ted.com/talks/pattie_maes_d la-fi-mo-autonomous-cars- emos_the_sixth_sense.html 20120521,0,331233.story An article on electronic contact The online Adobe Museum of digital me- lenses and their possible application dia, http://www.adobemuseum.com/ on augmented reality display, http://www.newscientist.com/blogs/onep On fabrication projects and communi- ercent/2011/11/electronic-contact- ties: lens-displa.html

The Fablab wiki, http://wiki.fablab.is/wiki/Main_Page

58 From Paper to Portable Devices: How to script and read comics? Jérémy RAULET and Catherine SAUVAGET and Vincent BOYER LIASD, University of Paris 8 2 rue de la liberté Saint-Denis, FRANCE, jraulet, cath, boyer @ai.univ-paris8.fr { }

Abstract is the era where paper atoms become pure in- Nowadays, reading comics is a common activity. formation: bits. To obtain a good numerical Due to the portable device advent and the wide representation we have to study the way used spread of their high-performance, the market of rich to proceed on paper. In this paper, we focus on content is enlarging. However, the adaptation of comics reading on portable devices. comics illustrations on such devices is not an easy Comics may have different forms varying from task: graphic novels should stay readable and en- a single strip to a book collection. Image se- joyable. Moreover, producing thousands adapted quences in graphic novels are sometimes non- comics is a long and fastidious work for designers. linear structured. Moreover, the different pic- We propose a system dedicated to help designers who generate digital content from existing comics tures have generally various sizes and it is a books for different portable devices. Our tool helps tricky task to adapt the way to watch them on designers in three ways. First, by extracting panels embedded devices. Visualization becomes a cru- using a binarization with propagation; second by de- cial problem and content delivery device should tecting overlaps with a path planning method; third be adapted. Even if the screen resolutions are by scripting comics reading intuitively and quickly, small, the user should have a global view of an providing a glyph-based interface. The first at- image or text, easily focusing on details through tempts show that extraction is 60 times faster with the tactile interface. However, reading a page our tool than designer performances and that our glyph method is 40 times faster than hand-made containing sequences of images on a smartphone transition. This tool provides a new and efficient or a tablet device remains a problem, because way to read comics on any mobile device. its global view does not provide any suitable resolution. Furthermore, as comics reading is Keywords often non-linear, finding a way to predict panel transitions seems difficult. That is why the Computer Graphics, Picture/Image Detection, reader only should have to focus on the story Graphical User Interface, Media Art, Comics. and should not have to suffer from the research of the next image. 1 Introduction In the industry, existing graphics novels on Nowadays, numerical softwares and tools renew mobile devices are created in two manually at an ever greater speed. With the advent of ways. The first one consists in creating images smartphones and the emergence of tablet per- dedicated to mobile device screens. The second sonal computers, many art forms may be found consists in scanning a comics page, then each on those new devices. They allow to transport image is extracted and sorted manually by a a lot of content and people read more and more designer using manipulation tools (Adobe Pho- numeric books, newspapers or numeric comics. toshop1 for example) that produce very limited Incoming purposes, new objects or materials results for a high cost process (i.e even if these can be conceived from the wide concept and tools permit to crop, resize. . . , they are not ded- practice possibilities of art [Reyes-Garc´ıa and icated to script the reading). If an image size is Zreik, 2008]. Each desire, invention or need cre- not appropriate, the designer has to cut it, to ates a cycle leading us to new technologic level. resize it or to create an animation that permits These new technological devices and their use to view the entire image. For both cases, de- come from the panel of social practices. Thus, 1 R R “a tool is the idea that art as information is in- Adobe Photoshop is a graphics editing program herently unlimited” [McCloud, 2000]. Our era published by Adobe Systems

59 signers have to create animations between the tions. In his book “The Invisible Art” [Mc- different images of the pages for a better reading Cloud, 1993], Scott McCloud has defined comics experience. as “juxtaposed pictorial and other images in de- This paper addresses the following questions: liberate sequence, intended to convey informa- How should be transformed comics to be eas- tion and/or to produce an aesthetic response in ily readable on an embedded device? Should the viewer”. Rodolphe Töpfferis considered as the page be prepared entirely automatically? If the first theorist of this art and as the father some operations should be realized manually, of the modern graphic novels [Groensteen and what kind of interface should be used? Peeters, 1994]. Scott McCloud has said that We propose a glyph-based tool to script Töpfferhas created a way of expression com- comics reading for different devices with the re- ing from drawing and writing that cannot be spect of their own screen size and resolution reduced to one of them: an autonomous lan- constraints. We attempt to propose a “remedi- guage [McCloud, 1993]. Thierry Groensteen has ation” [Bolter and Grustin, 2000] that is a hy- thought that “the verbo-iconic components of permediation where we are aware of the medium narration are inseparable” [Groensteen, 1990]. and an immediation allowing the medium to become transparent and confronting us to the con- A comics is created in several steps [Duc, tent: the comics. To make it possible, it is im- 1983]. The first step, named synopsis, consists portant that the successive panels of the comics in writting an abstract explaining the global appears in an easy way, as we forget the page scheme of the story. Then, the scenario de- use of comics books, totally engrossed by the tails the story, the scene, the principal charac- story. Our tool allows a global view of the page ters and provides the main developments. The to extract its images, to sort them, to produce next stage consists in cutting the scenario in various transitions between them (fades, move- different images that forms each panel. This ments, accumulations. . . ) and to display finally is probably the most crucial step of the cre- each image on the device screen with an anima- ation because if the scenario is well cut, with tion. Glyph-based interface is particularly well- rhythm, it can save a bad drawing. At the op- adapted to the scenarization of comics reading posite, good drawings would rarely save a poor since we think that this task should be realized cutting. The words in the scenario descrip- by a designer or one of the comics authors who tion are replaced by shots or frames, camera often use graphic tablets. Moreover, for each angle and centerings to compose the strip or target device, we provide a viewer able to read the different comics pages. A camera angle is the produced script and to allow user interac- chosen to render an atmosphere like anger or tions. Note that more than a comics reading defeat. For example, a high-angle shot put the scenarization, our solution can be used to script character in a weak position. A vertigo sen- any non-linear structured media. sation will be represented by a frame conveying this idea which probably will be vertical. First, we describe how a comics is created and The frame of each panel is dedicated to present we give some definitions. Then, we present ex- the story and can be very brief (an action, a isting researches and tools regarding the differ- glance. . . ) or take a while (a background de- ent steps to export a comics on portable devices. scription) and gives the timing of the reading. Then, we propose our tool dedicated to the de- If there are enough varying shots, monotony will signers who will create the animated comics for be avoided. They are never randomly chosen, portable devices. After that, we present some their main interest is the psychological and ex- results and discuss them. Finally, we conclude pressive value permitting to highlight charac- and propose future works. ter’s action or sentiments. They modulate the 2 Sequential art and definitions dramatic or humorous intensity of each scene following the plot. Consequently, each panel Graphic novels have been slow to be considered th 2 has a size, a dynamism, function of the shot, as 9 art . They are one of the most expres- and may have various forms: rectangular, cir- sive media with a huge style diversity, allow- cle, polygonal, undefined. Panels are often sur- ing the expression of various ideas and emo- rounded by black borders. In the following, the 2The 9th art expression exists since 1964 and comes term of open panel is used when the panel is from Maurice de Bévèreand Pierre Vankeer, creators of not clearly bounded by borders, on at least one Lucky Luke. edge. Panels can contain text often represented

60 in speech balloons or onomatopoeia. panel filling an entire page). In the main Comics pages are sometimes non-linear panel there are multiple images, with inclu- structured and can be difficult to read. Thus, sions, overlaps. Here, the text is not used to the page layout is very important and lets the depict dialog but to describe timing and inten- reader focuses on the story. In comics books, tions enhancing fear, anger and death filling. the panels are generally separated by blank rib- It is obvious that the panel size and overlaps bons of paper named gutters [McCloud, 1993]. that constitute the page layout were conscien- It guides the reader and can radically mod- tiously chosen by the comics creators to high- ify the perception of the story. Sometimes, a light and emphasize some story elements or to panel element overlaps another panel creat- depict the timing. ing a highlighted transition between these two As another example, we present in bottom of panels. figure 1 a page containing height open panels. The reader impression is the continuity of the action: panel sizes are approximately the same, so time for the action is the same in the entire page. Here, the text is mostly used for onomatopoeia which means that the text is used to describe the action (the speed of the chase). Comics is not just a series of panels on a comics pages. We want to propose comics reading on portable devices. Nowadays, comics dedicated to portable devices are specifically created for this kind of device or they are exported from old comics books. Automatized solutions to extract panels should be provided when there are no ambiguities between panels. The scripting of comics reading should be helpful for the comics creators on mobile devices. Today, in the industry, designers have to cut manually the different panels of a comics book and then to animate them. This task take a long time and the existing tools to extract, sort and animate panels are not dedicated to comics page layout. In the following section, we present existing tools to extract or scenarize panels with animations.

3 Previous works Due to the structure of a comics page (see section 2), and with a study of existing research Figure 1: Top left: Page 6 of Asterix and the Se- projects, we have identified three main prob- cret Weapon [Uderzo, 1991]. Top right: Page 3 lems to scenarize comics reading: of Star Wars: Legacy - War 1 [Ostrander et al., 2010]. Bottom: Page 2 of Kromagnons [Taloux Panel extraction of a comics page; and Chantoux, 2009]. • Identification and treatment of overlapped Left of figure 1 presents a comics page where • panels; almost all images are surrounded by borders, in this page the text is in speech balloons. Scenarization production by scripting the As one can see, the rectangular panels have • comics reading. different sizes and an open panel is present in the middle of the second strip. Due to the Hereafter, we present related work focusing overlap, it is obvious that the third strip is on comics. Finally, we focus on sketch-based read more continuously than the others. interfaces which are close to our scenarization Right of figure 1 presents a splash page (a problem.

61 3.1 Extraction have meaning, one cannot just cut them without Ishii et al. [Ishii and Watanabe, 2010] have used thinking about the page layout. For example, in density gradient and corner detection to extract left of figure 1, the overlap shows the character panels from comics pages. Their method is lim- irritation. ited to panel detection. Incomplete panels and We consider that automatic cut of overlaps open panels may not be detected. Moreover, does not always provide a good solution. Over- overlaps are treated as a single image. laps contain significant information linking mul- Arai et al. [Arai and Tolle, 2010] have pro- tiple panels (see top left of figure 1, third strip). posed a binarization with a fixed threshold and 3.3 Scripting comics reading then an extraction of single blobs. These blobs In “Infinite Canvas” [McCloud, 2009], McCloud are considered as panels. They assume that a has explained that digital comics have many ad- panel has a minimal size of a sixth part of the vantages. They have no restricted size, it can page width and an eighth part of its height. be an entire page or a single panel. So the flow These conditions are used to guide the detec- of reading is not broken by the page, the break tion and are not suitable for all comics pages. matches with the story break. Thus, the scenar- Ngo ho et al. [Ngo ho et al., 2011] also have ization is needed due to the page format used binarized a comics page but they use a region- in comics books. based image segmentation. They assume too, Arai et al. [Arai and Tolle, 2010] have pro- that a panel has a minimal size of a sixth part posed to export panels and extract speech bal- of the page width and a sixth part of its height. loon texts. Nothing is done to script the read- Here also, the minimal panel size is fixed to ing, except a simple arrow to move to the next avoid the detection of small zones. panel. This solution only consists in reading one The panel extraction is commonly realized panel, the panel transition is never considered. automatically. All these researches consider Wandani et al. [Wandani et al., 2011] have that panels have borders, so they use very sim- proposed a different way from panel to panel to ple solutions that do not take into account open read comics. In this paper a way to display a panels. Unfortunately, some panels (see bottom comics or a page in a single screen is proposed of figure 1) cannot be detected without loss us- using two prototypes: ing these solutions and the minimal size chosen remains a problem. 1. All comics panels are arranged in a sequence on the border of the screen. In the 3.2 Overlaps center, a zoom is applied on the selected Different solutions have been provided to detect panel (left of figure 2); and treat overlaps. Arai et al. [Arai and Tolle, 2. Panels are arranged according to the story 2010] have proposed a solution based on a sim- and displayed with a small size on the ple straight line drawn by the designer. In prac- screen. The user can zoom on the selected tice, the detected blobs are displayed and the panel to view it bigger and move to go to designer can specify which parts can be consid- the next panel. They have two different in- ered as overlaps. This approach is not sufficient terfaces for this prototype based on “Infi- since it is only based on a single straight line nite Canvas” [McCloud, 2009] (right of fig- cutting the overlapped images. Consequently, ure 2): overlap created by speech balloons for example are cut and the balloon is cut in both images. A “Page view” where panels of the • Ngo ho et al. [Ngo ho et al., 2011] have used same page are reorganized from the mathematical morphology to transform a block original page layout; representing panels. These transformations en- An “Overview” where all the panels able to break overlaps with a dilatation and ero- • are reorganized and displayed in the sion to restore the size of the black area obtained same screen. by binarization after the previous step. One of the method restrictions concerns the over- These two prototypes are interesting ap- lap size. The method produces convincing re- proaches but do not seem very usable with a mo- sults when the overlap size is small. Moreover, bile device. The first prototype lets a very few the approach neglects the semantic meaning of place to the current panel and the second forces the overlaps when cutting images. Overlaps the reader to manipulate panels every time.

62 Figure 2: Prototype proposed by Wan- dani [Wandani et al., 2011], respectively prototype 1 and 2.

As one can see, proposed solutions are automatic and basic. None of them considers the timing as an important element and the size of the panels is never adapted to the device. Also, this effect is disturbing because the page layout is completely removed. This should be preserved and described by a designer through a convenient graphical user interface (GUI). 3.4 Sketches Figure 3: Kinds of drawings [Gross, 2009]. Making sketch-based interfaces is a powerful way to express something and seems to be the most suitable solution to provide any scenar- izations of the comics reading. We think that scenarization must be realized by an artist or a designer who often use graphic tablets and who, consequently, use sketch very naturally. Apitz et al. [Apitz and Guimbretière,2005] have proposed a GUI named CrossY that enables to replace a button-based interface with a menu by a sketch-based interface. For Apitz, “crossing is as expressive as the more traditional point-and-click interface”. As Apitz, we want to provide a system as simple as possible in which sketches are used to select, organize, resize, cut and script. As shown by Gross [Gross, 2009], two classification systems can be used to depict a sketch- Figure 4: Kinds of systems [Gross, 2009]. based interface: kinds of drawings and kinds of systems. On the two axes of the “Kinds of systems” • diagram (see blue mark in figure 4) we can On the three axes of the “Kinds of draw- be located: • ings”diagram (see blue mark in figure 3) we can be located: – In the interaction part (like CrossY [Apitz and Guimbretière, – In the geometric part on the domain 2005]) on the descriptiveness axis axis describing the domain as geomet- representing the tasks for which pen ric or not; based interaction is used; – In the project part on the purpose – In the middle of the generality axis axis representing the different pur- indicating the degree to which a sys- poses that a drawing can serve; tem is intended or designed to support – In the diagram part on the denota- a specific domain. tion axis which represents the degree to which a drawing is directly denota- As described in the following, we use simple ge- tional. ometric glyphs to script comics reading.

63 4 Searching for an ergonomic (or test all corners), considering that it can be interface dedicated to designers noised due to digitization or that is potentially In the following, we present our glyph-based a panel positioned at the border of the page tool dedicated for designers and inspired by (without blank ribbon). Considering that the page layout considering three main problems background is almost uniformly ﬁlled we prop- (from top to bottom approach): agate the binarization on the map while the pixels have the background color. A thresh- 1. Allow the designer to extract the successive old is used to avoid noise due to the digitiza- panels of the comics, hereafter called page tion. During the decomposition, we create at decomposition; the same time polygonal bounding boxes. When two close neighbors bounding boxes have been 2. Allow and help the designer to specify the created, we automatically merge them. In prac- properties of each displayed panel (when tice we maintain a rectangular bounding box for and how it appears and disappears, reading each detected panel and we merge them when direction), hereafter called panel proper- they have at least 30% of intersection. Finally, ties; we export as panels all areas which are not bi- 3. Allow the designer to export it for sev- narized in the map. eral portable devices, hereafter called exported ﬁles and device viewer.

Concerning the page decomposition, we assume that panels can have various sizes. Thus, we do not fix a minimal one, but we use a threshold of 1% of the page (in number of pixels) to remove noise or page numbers for example. As previously described in section 3.3, there is no convenient solution to create a comics reading scenarization. Actually, in the industry, this process, is entirely achieved manually by a designer. We consider that scenarization should be comics friendly designed. Starting with comics pages, our glyph-based tool is designed to produce a description file and a set of panels that can be read on any portable devices with our viewer. Therefore, comics reading is composed by many screens. Each screen is composed by one or more panels. In case of multiple panels on a screen, we use hereafter the term of accumulation. The timing depends largely on the screen composition, the panel size and the presence of Figure 5: Top left: page 6 of Asterix and the Se- speech balloons. Moreover, it may be reinforced cret Weapon. Top right: its binarization. Bot- by the transitions between screens. tom: zoom on the open panel of the second strip. 4.1 Page decomposition and overlap detection Figure 5 illustrates this binarization. We pro- Helping the designer to create the different vide the original comics page on the top left and screens, as a preliminary step, we automat- the result of the binarization on the top right. ically extract panels by detecting the back- Remark that for the second panel of the second ground color used in the page. Our approach strip (zoom on bottom of figure 5), the open is based on a binarization process that creates panel is considered as one single panel by our a map of the background. The background system because of the two elements vicinity. color is detected at the page corner. For a cor- We consider that an automatic cut of overlaps ner, if only one panel is detected, we test an- is not a suitable solution. If an artist has drawn other corner until we find more than one panel an overlap, it is obvious that it is one element

64 of the comics page layout. Thus, we provide a (for example, left to right for a Franco-Belgian solution to detect overlaps to prevent the de- comics and right to left for manga pages). The signer. Considering its semantic meaning, the designer can reorganize panels sorting them by a designer will decide which treatment should be click and release action on each one or by draw- applied during the scenarization. ing a line passing through the different panels in The overlap detection is based on the detec- the order he wants them to appear (see bottom tion of incomplete panel frontiers on the bina- of figure 6). If the original proposed order is the rized map. We use a path planning algorithm good one, the designer can just validate it. to detect dead-end possibilities which are considered as an overlap consequence (i.e. two panels are not clearly separated). Small dead- ends (less than one third of the panel size) are neglected since we consider that noise can be present in the image due to the digitizing. Fig- ure 5 presents an overlap on the third strip. It is detected by the bottom frontier of the panels with a high level of confidence.

Figure 7: Global view of our glyph-based interface dedicated to script comics reading. Here, the triangle indicates that the panel disappears with a fade. After sorting, the designer sets panels properties like transitions or size. This is made through a glyph-based interface (see figure 7) where a glyph matches to an action (change mode, add an effect. . . ). First, we check if the glyph is closed to know if it is a closed form like a circle, or an opened form like a line or a combination of lines where the designer has to put up his hand to draw the next one. If the glyph is closed, we search lines in it and we count them to know what is this glyph. For ex- Figure 6: Top: page 38 of Fate Spinner. Bot- ample, a closed glyph with 3 adjacent lines is a tom: sorting the detected panels. triangle. The way of these lines might be im- Figure 6 is a resulting example of a page de- portant: 2 horizontal lines and 2 vertical lines composition. Remark that the girl alone and represent a box when 4 diagonal lines represent the speech balloon close to her have been ex- a diamond-shaped box. tracted automatically as a single panel. The screen interface is divided in 5 parts (see After this step, the designer can cut panels. in figure 7): If he desires he can cut an overlap by tracing a At the top left, we provide the panel order line where the cut should be. Then, he can sort described above. This is used to select a and script the comics. panel to treat. It is also used to combine 4.2 Defining panel properties two or more panels to be displayed with an This part of our tool (figure 7) is based on a accumulation; low level recognition of simple glyphs. It allows The bottom right area is used to select the the designer to create a scenarization for comics main area mode (see hereafter). For each reading on different portable devices. mode, we give its associated glyph: First, the designer can modify the panel order following his desires. This step cannot be – In-coming ( ): panel animation ap- automatized due to the comics style variety plied when the→ panel appears;

65 – Out-coming ( ): panel animation ap- virtual device screen in landscape format is plied when the← panel disappears; displayed through lines: – In-screen ( ): if the panel oversizes the · – A green line indicates that the panel screen device, then the designer can can be displayed entirely on a device; choose the panel movement; – A red line mentions that the panel is – Position ( ): panel position on the de- 4 cropped due to its size, position or vice screen; both. The designer may specify in this – Size ( or ): used to resize automat- case an in-screen movement; ically or manually the panel if neces- 2 3 sary. In the middle area on the right, we produce information summarizing the scripted ac- At the bottom left, we find the main area. tions. For example, we can view, if a panel Here, a panel can be displayed and we use appears with a fade, a rotation or both; glyphs to describe one or several actions In the top right area, we provide three depending on the selected mode (area ): buttons. The “Export” button is used to – Panel animations (in-coming, out- produce the xml file (see section 4.3), the coming): we provide a collection of “Help” button summarizes the glyphs and various possible animations defined by the last button is used to switch to a clas- a glyph for each of them: fade ( ); sical “Button” interface or to our glyph. flash (+); zoom-in ( ); zoom-out (4); When the designer has finished his work, he rotation, clockwise and2 counter clock-3 uses the “Export” button that produces the wise ( ); flicker (– – –); 8 directions description file. This file permits to read ( ,..., ); the comics on any portable device which ← - – Panel size and position (in-screen has a viewer. movements, position, size): based on 4.3 Exported files and device viewer comics structure analysis, we provide We provide a description file per comics which 8 reading directions ( ,..., ) and contains the properties of each extracted panel. any zigzags with only two← parallel- lines As a consequence, one can easily imagine to up- (N, Z. . . ); load his comics scenarization on different de- – Panel combination (depicting accu- vices without any modifications. For each de- mulation of multiple panels and device on which we desire to read a comics, we ciding when they disappear): the de- need a viewer (see figure 8 for an example). This signer can encircle consecutive panels viewer differs from an architecture to another. to provide panel accumulation on the screen. It is achieved in . Therefore, panel transitions defined for exit are performed when the first panel which is not in the accumulation appears. Multiple effects can be combined for a transition. For example, a panel can appear from bottom right to top left with a rotation and a zoom. In this case, the panel, which is out of the screen at the beginning, Figure 8: A panel in iPhone viewer. See addi- moves from the bottom right to the posi- tional material for a video. (http://youtu.be/ tion defined by the designer. The panel sGYOuIGGveg) may also grow in continuous and linear pro- portions until the desired size. Concur- The viewer permits to compute and to gen- rently, the panel rotates around is own cen- erate transitions, accumulations and to adapt ter. sizes and positions of each panel. It must also If we create an accumulation in , when we display automatically panels and allow the user display a new panel, the previous panels are to zoom or to move manually forward or back- displayed too, with a low transparency. A ward. We also allow the user to disable panel

66 transitions to respect the original immobility of out sound) in three minutes (depending on the the comics. speed of drawing and the complexity of the desired script). 5 Results A video is given as additional material (http: This work is a collaboration between a com- //youtu.be/sGYOuIGGveg) and presents our puter graphics laboratory and a producer of tool and the glyph based interface in details. entertaining applications. For each page, we In this video, we do the scenarization of a page calculate the time taken by our system and of the manga One Piece [Oda, 2012]. the time needed by a designer to realize the Our tool is dedicated to the industry. Be- same tasks. Table 1 present results of ex- cause of the the right of the firm, only the de- traction and scenarization computed on 273 signers tested our system. After fifteen minutes pages (from comics, manga and Franco-Belgian everybody find it easily to use and very intu- comics). These pages have different configura- itive. They found a real interest in this tool and tions: from one to thirteen panels, with or within particular the saving time for extraction and out borders, including overlap or not. Digitizing scenarization. However, they emit a small crit- the pages, noise can appear on some of them. icism on the scripting part: they would like to define glyphs themselves. For example, one of who Designer Our tool them would like the flash effect to be a lightning time bolt glyph ( ). More generally, they would like extraction 2 minutes 2 seconds ≈ more possible effects: checkerboard, scanning, scenarization 2 hours 3 minutes strip. . . Table 1: Time comparison between our tool and a designer: average time per comics page. 6 Conclusion We have proposed a glyph-based tool to script intuitively comics reading for different devices 5.1 Extraction with their own screen size and resolution. To Our extraction process detects correctly 96% of ensure that comics are still readable on portable the images if we consider overlapped images as devices and with respect for sequential art na- a single one and 80% in the other case. Remark ture, we extract panels from comics pages. We that compared to a process realized by a de- provide an automatic page decomposition sys- signer on an image manipulation tool, where 90 tem detecting panels and overlaps. Indeed we minutes are necessary (2 minutes per page), we have shown that an entire automatic solution is provide a solution to extract panels on 44 pages not desirable due to comics diversity and the se- (classical format for a Franco-Belgian comics as mantic meaning of each element. In regards of in [Uderzo, 1991] for example) in 90 seconds at previous work, we provide a glyph-based inter- worst. It is crucial to recognize and preserve face which is very well adapted to the graphic the meaning of open panels. Bottom of figure 5 tablets often used by designers and artists. Our gives an open panel example. For 73% of open interface allows them to sketch easily all panel panels, we produce only one panel and do not properties. We also provide a button-based so- cut it. lution to reach a wider public and we propose in Binarization is easy but it is the essential part future work the scripting of various media. Our of the extraction and it enables to separate pan- tool provides a complete solution to extract and els more efficiently. However, the binarization script comics reading in few minutes per pages is not a “complete solution”. For example, the while other solutions take at least several hours. third strip of figure 5 has a gutter half binarized More than a comics reading scenarization, because of an overlap. The rest of this page was our tool is designed to script any non-linear correctly cut. structured media and for example one can easily imagine to create efficient slideshows. It is 5.2 Scenarization also conceivable to use our tool to transform In our tool (see figure 7), glyphs are used the pages of newspapers and script article read- to script comics. Today, in the industry, a ing for tablet personal computers. To do this, comics page is scripted in two hours (adding we only need to change the page decomposition sound effects and music). Our solution en- proposing a choice between comics and news- ables to achieved this process (currently with- papers at the launch of the tool. Newspaper

67 decomposition could be done encircling articles S. McCloud. 2009. The “Infinite Canvas” with a sketch-based interface using our glyph Ideas. recognition. A. K. Ngo ho, J-C. Burie, and J-M. Ogier. In future work, we plan to manage text posi- 2011. Comics page structure analysis based tions and speech balloons to improve the comics on automatic panel extraction. 9th Inter- reading. Finally, we desire to detect panel char- national Workshop on Graphics Recognition acteristics (size, form. . . ) to automatically pro- (GREC 2011), September. pose reading directions and transition effects to the designer. E. Oda. 2012. One Piece: The Road Towards the Sun. Glénat. 7 Acknowledgements J. Ostrander, J. Duursema, D. Parsons, and This work was supported by TEKNEO B. Anderson. 2010. Star Wars: Legacy - War (producer of entertaining applications (video 1. Dark Horse Comics. games), as well as serious applications (Serious games)). Jeremy Raulet was a recipient of a E. Reyes-Garc´ıaand K. Zreik. 2008. A social CIFRE fellowship. hyperdimension of Media Art. International Journal of Design Sciences and Technology. References Taloux and D. Chantoux. 2009. Kromagnons. G. Apitz and F. Guimbretière. 2005. Tekneo. CrossY: A Crossing-Based Drawing Applica- tion. ACM SIGGRAPH. A. Uderzo. 1991. Asterix and the Secret Weapon. Orion. K. Arai and H. Tolle. 2010. Automatic E-Comic Content Adaptation. International A. D. Wandani, G. Wee, and W.S. Moses. Journal of Ubiquitous Computing (IJUC), 2011. Designing Interactive Mobile Comics 1(1):1–11, November. for Multi-Touch Screen Phones. International Conference on Future Information Technol- J. D. Bolter and R. Grustin. 2000. Remedi- ogy IPCSIT, 13. ation Understanding New Media. The MIT Press. B. Duc. 1983. L’art de la BD. Glénatet B. Duc. T. Groensteen and B. Peeters. 1994. Töpffer : L’invention de la bande dessinée. Collection Savoir: Sur l’art. Hermann. T. Groensteen. 1990. Au commencement était töpffer. Le Collectionneur de bandes dessinées, 64:10–21. M. D. Gross. 2009. Visual Languages and Vi- sual Thinking: Sketch Based Interaction and Modeling. Workshop on Sketch-Based Inter- faces and Modeling 2009. D. Ishii and H. Watanabe. 2010. A Study on Frame Position Detection of Digitized Comics Images. Workshop on Picture Coding and Im- age Processing (PCSJ2010/IMPS2010), De- cember. S. McCloud. 1993. Understanding Comics: The Invisible Art. Tundra Publishing. S. McCloud. 2000. Reinventing Comics: How Imagination and Technology Are Revolution- izing an Art Form.

68 Designing Natural User Interfaces with Depth Sensing Technologies like the Kinect Sensor - A Tracking Framework for Artists and Designers

Michaela Honauer Prof. Dr. Jens Geelhaar Faculty of Media, Interface Design, Faculty of Media, Interface Design, Bauhaus University Weimar Bauhaus University Weimar Marienstraße 5 Marienstraße 5 Weimar, Germany, 99423 Weimar, Germany, 99423 [email protected] [email protected]

Abstract sensors work in order to create user-friendly This paper presents KinectA, a tracking application interfaces and projects that make use of this for media artists. The potential of natural user technology for new paths in human computer interfaces (NUI's) via depth sensing technologies is interaction. Therefore this paper gives an insight discussed. Recent literature on the definition of into Kinect's operation mode and the possibilities NUI's is summarized. The Kinect Sensor is to get access to the Kinect via existing frameworks. identified as a successful example for the Finally, we present a prototype of a tracking technology behind NUI's, the operating mode is application via depth sensors that is especially explained and the major coding frameworks are created for media artists and designers who want to introduced. focus on their creative work. This software enables them to track persons, hands or objects and to use Keywords and process the tracking results in their individual Natural User Interfaces, Kinect Sensor, Depth- projects. Sensing Camera, 3D Tracking, Designing Natural Interaction. 2 About Natural User Interfaces

1 Introduction 2.1 A Literature Review We believe that the qualification of natural user The definition of natural user interface is mainly interfaces (NUI's) plays an important role in the described and promoted by Microsoft researchers field of virtual reality, augmented reality and [3]. In their sense the term natural is associated ubiquitous computing when the difference between with an easy and effective path to interact with reality and fiction disappears for the interacting technology. They say it implies a quick and users. Thereby a user interface becomes more and enjoyable learning process for novices as well as more natural if it enables its users to operate with for non-experts, because NUI's do not necessarily computer systems in the way they interact with require a special instruction nor artificial input their everyday environment [4]. During our devices (e.g. mouse or keyboard) [8, 12]. They research we noticed that in the early 2000's efforts tend to result in an intuitive experience and “enable in the field of prior natural interaction research users to interact with computers in the way we focus mainly on gestural interfaces which are interact with the world”1. But it is not an effort to clearly dominant in the field of natural interaction copy the real world, as Wigdor and Wixon point until today [1, 5, 8, 9, 10]. out, natural refers rather to a feeling and behavior The technology behind a NUI is currently often users have during the interaction [12]. connected to depth sensing technologies. The Further definitions of NUI's are discussing the Kinect Sensor (abbreviated Kinect) is probably the seamless natural behavior towards the interaction most popular one. It was primarily thought to be a with technology [3, 4, 10]. In this sense the controller-free gaming tool but nowadays it is also technology behind becomes almost invisible for being used for the purposes of medicine, the user. Some authors even state that the NUI education, science, business, fine arts and other itself is not visible. Saffer describes it as cultural issues. “interfaces without faces“2. But this point of view

It is important for media artists as well as for 1 computer scientists to understand how such depth Jain et al, p. 211. 2 Saffer, p. 147.

69 appears not to be fully accepted because other Concluding this literature review, the subsequent authors do not agree. They rather believe that “the characteristics can be summarized: idea of the NUI as a fully tool-less interface is – At first a NUI is the contact point between overstated“3 and invisibility associates that no man and machine that does not require interface representation is needed as it is in artificial input devices like physical comparison to traditional graphical user interfaces controllers. (GUI's) [12]. Concurrently it is also not excluded – Second it enables users to interact with that NUI's could be visually represented [2]. technology through body movements, The differentiating characteristic is the novel speech and sound or other biophysically interaction mode of NUI's which is related to a given capabilities that are ideally more intuitive behavior and easier experience for combined to a multi-modal interface. the user, which today is mostly based on gestures, – Third the technology behind a NUI is but which becomes more and more natural if it almost invisible. supports a combination of all human expression 2.2 The Potential of NUI's modes (e.g. seeing, gestures and facial expression, hearing and speech, tactile sensing, senses of taste We will now shortly discuss how natural user and smell) [4, 12]. Nielsen et al highlight this, too interfaces assert themselves in the near future in [5]. They have developed and tested a procedure different cultural and non-cultural areas. NUI's are with a selected set of gestures. Their experiments “another potential evolution in computing“9 revealed that “gesturing concerns more than just a although gestural interaction is currently their hand posture or movement. It affects a greater part predominant mode. But they will not replace of the body, biomechanically speaking“4. traditional interface paradigms like GUI's because If a NUI is just based on gestural interaction it is this type of interface is widespread and anyhow not natural [4, 5, 7]. Donald A. Norman sophisticated [11, 12]. Nielsen et al emphasize for argues that most “gestures are ephemeral“5 and not instance that a “gesture interface is not universally “easy to learn or remember“6. In addition just a few the best interface for any application“10. NUI's will “are innate and readily predisposed“7. Norman and probably fill computational niches and they “will Nielsen later insinuate that the different companies bring unprecedented interaction experiences and in the industry of gestural devices “use different capabilities to computing”11, similar to how rules, which end up confusing people”8 and show, graphical user interfaces brought new interaction on the basis of several fundamental principles (e.g. modes to the age of the command-line interfaces. visibility, feedback, consistency, scalability and Although Donald A. Norman is on the one hand reliability) of interaction design that currently no skeptical caused by the missing standardizations, standards for gestural conventions are existing [6]. he sees gestural systems on the other hand as “a The phenomenon of missing standardization valuable addition to our repertoire of interaction could be caused by the fact that the analysis of techniques“12. Gestural behavior holds some gestures was and particular still is not standardized advantages for the interaction with technology. [9]. That point of view is connatural to the Gestures can for instance be tracked over distance argumentation of Brockmann and Müller. They or in noisy environments when voice input or input state that gestures are often redundant or with artificial control devices is not possible [1]. spontaneous [1]. We assume that cultural At least in the gaming industry, Kinect mediated differences have to be considered additionally NUI's seem to revolutionize the market. The because nodding the head does not mean yes in annual report of Microsoft Corporation (MS) every country, for instance. Therefore intended proves that they have sold 8 million sensors and gestures are hard to understand for a technical 6.3 million Xbox consoles in the second quarter of system or tricky to reconstruct with scientific fiscal year 2011 [13]. But they have only sold 5.2 methods [1]. That applies even in a more complex million consoles a year earlier when Kinect was way in multi-modal situations when gestures, not yet launched. This supports our opinion that facial expression and speech are combined. games with a natural interaction mode are 3 Goth, p. 14. successful. 4 Nielsen et al, p. 418. 5 Norman, p. 6. 9 Wigdor and Wixon, p. 5. 6 Ibid. 10 Nielsen et al, p.409. 7 Ibid. 11 Seow et al, p. 4768. 8 Norman and Nielsen, p. 49. 12 Norman, p. 10.

70 3 Depth Sensors as Media for NUI's reliable if the tracking stage is clear set up with a homogenous environment and with few interacting 3.1 The successful Story of the Kinect Sensor people who can appear and disappear in the tracking field without large difficulties. But up to A huge hype around Kinect Sensor takes its now we have not tested tracking sceneries with lots course since it has been released in November, of people, with a huge fluctuation rate, or under the 2010, by MS. It was meant to be a controller-free conditions of other human specified circumstances medium for the gaming console named Xbox. Just (e.g. with children or physically handicapped a few days after its market launch in the United people). States of America, the first open source driver was published. This was the birth of the OpenKinect 3.2 The Operating Mode of Depth Sensors community and the beginning of lots of So, how does Kinect finally work? The infrared experimental projects that made use of the Kinect spot and the infrared camera at the front side of the Sensor for innovative human computer interaction device build the starting point of the common in a way, that was probably not originally planned workflow. The diode transmits a pointed by MS. interference pattern that changes dynamically. This The employment of Kinect is so versatile that the pattern is reflected through humans and objects creative projects that already exist only give a inside the active space. The infrared sensor small insight in the multiple possibilities of perceives that reflection systematically and sends different application fields. But the question why the given data to the integrated chip. Afterwards this audiovisual depth sensor has become so the system on the chip calculates the information to popular still remains unanswered. In the previous a depth view. The processed information is again chapter we have collected the attributes of a natural sent to a connected PC or gaming-console. This user interface. Applications that make use of method is called Light Coding. So far the core Kinect's technology are mainly formed by all these concept of the system on the integrated chip is not qualification criteria: most of them use gestures published. For us it remains just a speculation how and speech, require no additional input devices and the depth information is finally computed. the technology behind is almost hidden despite the It is supposed that the depth information can be depth sensor and a processing unit in the calculated exactly for every illuminated centimeter. background. Inferential this medium comes along Persons or objects can be tracked up to 400 with an innovative concept for NUI applications. centimeters away from the device. If they are One can ask if the hype is truly justifiable and if further away the tracking results show inaccuracy. other technologies are not able to deliver similar All theses details are based upon environmental results for diverse project ideas. Indeed there exist conditions as already explained in the previous other depth sensing video systems but these sub-chapter. technologies are usually very expensive. In contrast the Kinect Senor and similar devices like 3.3 Three major Coding Frameworks the X-tion Pro by ASUS are absolutely affordable In order to connect the Kinect Sensor to a for most of the interests. computer three major frameworks exist. These are The further comparison with other depth sensing –the Microsoft Kinect for Windows SDK, tracking systems reveals that most of the other –thelibfreenect wrapper of the OpenKinect systems need an elaborated and complex Community and installation that might even require specially –the OpenNI framework trained experts. These systems are not as flexible The Microsoft programming framework has as the Kinect Sensor which can be easily plugged been released in February, 2012, and is only in and out, which enables mobility through its available for solutions on Windows based relatively small size, and which does not need an operating systems. This SDK (Software especially equipped or shielded room. Development Kit) is commercial software and We have tested the Kinect Sensor in different offers different features for skeleton tracking, face situations, inside and outside, under the conditions and speech recognition, object and gesture of natural light and artificial light as well as in the detection, as well as for hardware control (e.g. dark. It always worked excellent in our settings as change the LED’s color or tilt the motor). It long as there was no irritating infrared source like supports the programming languages C++ and C#. it could be caused through additional infrared spots OpenKinect is an open source community that or during the sunset when the infrared radiation of works on a non-commercial basis and disposes the light spectrum is very high. The Kinect is also open source libraries since Kinect's market launch

71 in the end of 2010. This solution is focused on the group it is important that they can choose between hardware access to Mac OSX, Linux and different tracking options (e.g. skeletons, hands or Windows. It supports a lot of programming objects) depending on their individual project languages such as C++, C#, Python, Action Script ideas. Furthermore, it is necessary for them to 3, JavaScript, Processing and some others. This control the tracked data and to decide which framework provides features of hardware control, information are involved in their creative access to the cameras and microphones, depth processes. visualization and detection, and it allows the use of multiple Kinect devices simultaneously. But currently it does not contain higher-level solutions like skeleton tracking. The framework of OpenNI (Open Natural Interaction) is an industry-led organization that publishes its framework for free. PrimeSense from whom MS has purchased Kinect’s technology is developing this SDK together with other companies who work on modern hardware and software solutions for new ways of human computer interaction. It is currently available for the languages C and C++, and can be used under Figure 1: KinectA in the workflow of media the operating systems of Mac, Linux and artists and designers. Windows. It offers hand and skeleton tracking, a set for gesture detection, speech recognition, 4.2 The Main Features limited hardware control, as well as the combination of multiple devices. KinectA is built on the framework of OpenNI and supports both skeleton tracking (see figure 2) 4 KinectA – A Tracking Application for and three-dimensional object tracking (see figure Media Arts 3). Additionally KinectA provides a simple hand- tracking feature (see figure 4). It is possible to run 4.1 The Motivation for this Project all tracking modes simultaneously. But the number of simultaneously tracked items is limited for When we started our research in the beginning reasons of computer performance: KinectA can of 2011, there was no tracking software available track in sum eight hands, four skeletons and 20 that met the requirements of media artist and objects at the same time. The basic settings contain designers who wanted to realize projects with the options such as activating the camera and changing Kinect technology. Most of the existing projects between depth, infrared and no view. By default that use depth sensors like the Kinect have their the depth view is on. own tracking concept. The tracking methods are integrated and not available separately. That means, they do not use an external tracking software for the access of the depth data. Although some of these projects have published their source code it is difficult to use the individual tracking methods for other project ideas. This is especially relevant for media artists and designers who often do not have sufficient programming skills for advanced and individual technical problems. They normally also prefer to concentrate on their creative work. In contrast the main idea of KinectA is to provide a tracking application that is just ready to run. Figure 1 shows how KinectA is integrated in the workflow of media artists and designers. This software is especially meant to meet the Figure 2: Skeleton-tracking with KinectA. requirements of media artists and designers, and assists them in their creative projects. For this user

72 KinectA knows four different states for the personal configuration (save, load, reset and new). In order to optimize resources the tracked information is not forwarded by default. KinectA offers an easy way to send a set of data. So users can choose manually which information they want to send via OSC (Open Sound Control)tothe network connection if the corresponding tracking function is activated. The network configuration can as well be changed individually.

5 Closing Remarks This paper has shown that depth sensors are innovative media for natural user interfaces and that the Kinect Sensor is currently the most popular Figure 3: Object-tracking with KinectA. one. It has drawn huge public attention from the beginning of its introduction. The interest of the open source communities has become as high as the interest of commercial institutions. It has been adapted for different issues and applications that experiment with new methods of human computer interaction. The examples for project ideas range from arts and culture over business applications to areas of science and medicine. The source code of lots of these project examples is published. But the availability of real tracking applications that are just ready to run is limited. Especially media artists have often not the hard coding skills to build an own tracking module for their projects and rely on the existence of support. Additionally we have explained the operating mode of this depth sensing technology. The coding frameworks that exist to access and process Figure 4: Hand-tracking with KinectA. Kinect’s depth sensing data are in general highly developed. Some frameworks are commercial Object tracking is implemented with others are free available. But not all of them considerable functionalities and allows additional support high-level solutions such as skeleton settings for processing the received image. It is tracking as it is implemented for instance in possible to define the optimal distances and sizes KinectA. of objects for a specific scenario, then capture a This software has been developed especially for static background image and remove it from the the needs of media artists who want to create final tracking images. Afterwards only new objects projects for natural interaction modes. It is a are considered for object tracking. In addition this tracking application based on the Kinect Sensor tracking mode provides some image filters for that offers simultaneously skeleton, object and smoothing, blurring, noise-reduction and hand tracking. The individual configuration of this intensifying of areas with weak signals. Object program can be saved for a later usage and the tracking is mainly based on the framework of tracked data can be forwarded to other hard- and KinectCoreVision but additionally allows the software via OSC. calculation of three-dimensional data of tracked KinectA is currently a stable software release objects. This helps to exactly define an object's that can be downloaded for different operating position inside a three-dimensional space. systems [14]. Its core concept is complete from our If users need the configuration of a scene several perspective but will be optimized through practical times our Kinect tracking application is able to tests and through the detailed feedback of its users save specific settings and configurations. The setup in the future. The underlying coding framework is is thereby referenced in a simple XML file. also in a prototype state. We will continue to

73 optimize KinectA in the sense that it becomes Revised Papers, pages 124-131. Springer- more comfortable for artists and designers to create Verlag, Berlin Heidelberg New York. prototypes and artworks for natural interaction. [10] D. Saffer. 2007. Designing for Interaction: Creating Smart Applications and Clever References Devices. AIGA Design Press, Berkeley. [1] C. Brockmann and H. Müller. 2004. Remote [11] St. Seow, D. Wixon, S. MacKenzie, G. Scott, Vision-Based Multi-type Gesture Interaction. In A. Morrison and A. Wilson. 2009. Multitouch A. Camurri and G. Volpe, editors, Gesture- and Surface Computing. In CHI EA’09 Based Communication in Human-Computer Proceedings of the 27th international conference Interaction - 5th International Gesture extended abstracts on Human factors in Workshop, Selected Revised Papers, pages 198- computing systems, pages 4767-4769. 209. Springer-Verlag, Berlin Heidelberg New Association for Computing Machinery, New York. York. [2] G. Goth. 2011. Brave NUI World. In [12] D. Wigdor and D. Wixon. 2011. Brave NUI Communications of the ACM, Volume 54 Issue World. Morgan Kaufmann Publishers/ Elsevier, 12, pages 14-16. Association for Computing Burlington. Machinery, New York. [13] http://www.microsoft.com/investor/Earnings [3] M. A. Hearst. 2011. ‘Natural’ Search User AndFinancials/Earnings/SegmentResults/Enterta Interfaces. In Communications of the ACM, inmentAndDevicesDivision/FY11/Q2/performan Volume 54 Issue 11, pages 60-67. Association ce.aspx (2012-09-26) for Computing Machinery, New York. [14] http://kinecta.mihoo.de/ (2012-09-26) [4] J. Jain, A. Lund and D. Wixon. 2011. The Future of Natural User Interfaces. In CHI EA’11 Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems, pages 211-214. Association for Computing Machinery, New York. [5]M.Nielsen,M.Störring,Th.B.Moeslundand E. Granum. 2004. A Procedure for Developing Intuitive and Ergonomic Gesture Interfaces for HCI. In A. Camurri and G. Volpe, editors, Gesture-Based Communication in Human- Computer Interaction - 5th International Gesture Workshop, Selected Revised Papers, pages 409-420. Springer-Verlag, Berlin Heidelberg New York. [6] D. A. Norman and J. Nielsen. 2010. Gestural Interfaces: A Step Backward in Usability. In interactions, Volume 17 Issue 5, pages 46-49. Association for Computing Machinery, New York. [7] D. A. Norman. 2010. Natural User Interfaces Are Not Natural. In interactions, Volume 17 Issue 3, pages 6-10. Association for Computing Machinery, New York. [8] J. Raskin. 2000. The Human Interface: new directions for designing interactive systems. ACM Press/ Addison-Wesley, New York. [9] N. Rossini. 2004. The Analysis of Gesture: Establishing a Set of Parameters. In A. Camurri and G. Volpe, editors, Gesture-Based Communication in Human-Computer Interaction - 5th International Gesture Workshop, Selected

74 !"#$%&'&()!*%+!%&%!"#$%,#*'!-".+%"-/!0*1,%(!#&%"*#"! ,"$!#2*",3!+,(,"*'! 2,",%(

444

!"#$"%!$&"'$()$*)+),$"-$!.)$" )$+.)'$"%!-!-$/0)-!1'),$!.)$#) !2"&3$145$"',$!.)$6%172!3$()!+))'5$"%!$ "', $ ')+ $ !)&.'1#1 38 $$9.)3 $ .", $ !1$ #)"%' $ !.) $ 0-) $ 14 $ ')+ $ !11#-5 $ 6%1 %"2- $ "', $ 2)!.1,-5 $ "', $ "&/0"'!$ !.)2-)#*)-$+!.$!)&.'&"#$)76)%!-)$:6%1 %"22)%-$"',$)' '))%-;$&1##"(1%"!1'-<$ =!.$61-!>, !"#$"%!5$?$2"@)$!.)$.361!.)--$!."!$!.)$#) !2"&3$14$"%!A-$%)#"!1'-.6$+!.$')+$!)&.'1#1 3$ :+.&.$."-$())'$*"#,"!),;$-$'1$#1' )%$!.)$&%0&"#$--0)5$"',$-.4!$!.)$41&0-$1'!1$!.)$'!)%%1 "!1'$14$!.- $ .3(%,A- $ -6)&4& $ &."%"&!)%-!&-8 $$B"'3 $ "%!-!- $ '1+ $ 0-) $ ')+ $ !)&.'1#1 )- $ 41% $ !.)% $ +1%@-C $ ! $ - $ "'$ 0')7&)6!1'"#$-!0"!1'8$ ?'$!.-$+1%@5$23$'!)'!1'$-$'1!$!1$,)-&%()$"'$1&&0%%)'&)$14$!.-$.3(%,!3$(0!$!1$%)4#)&!$1'$.1+$!.-$(%)),$ 14$6%1,0&!1'$&"'$()$6%14!"(#)C$.1+$!.)$4"&!1%$14$"%!$&"'$()$0-),$!1$'!)%)-!$6)16#)$+.1$"%)$"6"!.)!&$ !1+"%,-$')+$!)&.'1#1 )-8$$?'$1!.)%$+1%,-$+."!$&10#,$()$!.)$-1&"#$61-!1'$14$, !"#$"%!D 9.-$!)7!$6%161-)-$"$0'/0)$+"3$14$0-' $&%)"!*)$!11#-$!1$16)'$06$"$61-!$, !"#$2)!.1,1#1 3$E$.1+$+)$&"'$ '*!)$'1'>!)&.'&"#$-!0,)'!-$!1$)'!)%$!.)$,12"'$14$')+$!)&.'1#1 3$"',$26"%!$!.)$-@##-$')&)--"%3$!1$0-)$ '!)%)-!' $!11#-$'$!.)$40!0%)8

1 Teaching digital writing: $ ?'$!.)$-)&1',$-)2)-!)%5$?$.",$!.)$1661%!0'!3$!1$!)"&.$"$&#"--$1'$, !"#$+%!' $!1$"66%172"!)#3$!+)'!3 $ B"-!)%$-!0,)'!-8$$B1-!$14$23$-!0,)'!-$&"2)$4%12$!.)$F!)%"%3$9%"'-#"!1'$,)6"%!2)'!$"',$-12)$1!.)%-$4%12$ "$,)6"%!2)'!$"(10!$ !"#$?--0)-8$$=!.$4)+$)7&)6!1'-5$!.)$1*)%+.)#2' $2"G1%!3$14$!.)-)$-!0,)'!-$.",$'1$ ("&@ %10',$'$&1260!)%$-&)'&)8$$ $ 9.)$&#"--$&0%%&0#02$+"-$&1'-!%0&!),$1'$!.%))$6%'&6"#$"7)-H • I%1*-1'$14$"$&0#!0%"#$&"6!"#$!1$0',)%-!"',$!.)$'"!0%)$14$, !"#$#!)%"!0%)$!.%10 .$!.)$6%)-)'!"!1'$14$"$ &1%60-$14$+1%@-8$$J12)$)7"26#)-$14$+1%@-$-.1+&"-),$&"'$()$410',$1'$!.)$K10!0()$&."'')#$LJ ')-5$41%2-$ )!$M 0%)-N8 • P$!.)1%)!&"#$,-&10%-)$("-)$!1$",$!.)$-!0,)'!-$'$1(-)%*' 5$)*"#0"!' $"',$&%!&Q' $!.)$+1%@-5$"',$"$ #1("# $ 6)%-6)&!*) $ 14 $ , !"# $ +%!' $ )'"(#' $ !.)2 $ !1 $ 21*) $ !1+"%,- $ " $ 21%) $ 2"&%1 $ *-1' $ ' $ !.)%$ 1(-)%*"!1'-8$ R$$$$$$S%)"!*)$+1%@-.16-$+.&.$"##1+),$!.)$-!0,)'!-$!1$)76)%2)'!$+!.$,44)%)'!$"-6)&!-$14$, !"#$&%)"!*)$ +%!' $+.#)$)'&10%" ' $!.)$)76"'-1'$14$!.)%$2" '"!1'$ $ 9.)$&#"--$&126%-),$14$TU$#)--1'-5$)"&.$#"-!' $U$.10%-8$$9.)$#"-!$#)--1'$+"-$,),&"!),$!1$!.)$6%)-)'!"!1'$14$ )*)%3$-!0,)'!A-$4'"#$6)&)$14$+1%@$:)"&.$-!0,)'!$.",$!1$&%)"!)$"'$1% '"#$!)7!$!1$*"#,"!)$!.)$&#"--;8 9.)$1!.)%$TV$#)--1'-$+)%)$,*,),$'!1$U$&3&#)-8$W"&.$&3&#)$+"-$&1261-),$14$U$-)--1'-$"',$)',),$+!.$" $ &%)"!*)$+1%@-.168$P$+1%@-.16$() '-$+!.$!.)$6%)-)'!"!1'$14$!.)$)7)%&-)$!1$,15$!.)'$!.)$-!0,)'!-$+1%@$1'$ !8$P!$!.)$)',5$-12)$*1#0'!"%)-$6%)-)'!-$!.)%$+1%@8 $ 9.)$4%-!$&3&#)$+"-$&1'-)&%"!),$!1$!.)$!.)2)$LF)$!)7!)$G0-!)$#)$!)7!)DN$: !"!#$"!$%!&;8$$?'$!.)$() ''' $ #)--1'-5$60%)#3$!)7!>41&0-),$, !"#$+%!' $6%1,0&),$+!.10!$2" )-$+"-$)76#1%),5$+!.$"$6)%-6)&!*)$14$!.)$

1 ht/ tp:/ www.youtube.com/user/signesformesfigures?feature=results_main

75 #'@-$!."!$&10#,$()$&%)"!),$'$!.-$2'2"#-!$-)!!' 8$$P21' $1!.)%$)7"26#)-5$+1%@-$4%12$X)"*3$',0-!%3$ -0&.$"-$'()*!(+$+)%)$#11@),$"!8 P!$!.)$)',$14$!.)$&3&#)5$!.)$-!0,)'!-$.",$!1$&%)"!)$"'$"'2"!),$!)7!$+!.$I%)Q8 $

1.1 Prezi:

I%)Q$-$"$ ,>("-),$+)($-)%*&)Y-14!+"%)$!."!$-$0-0"##3$0-),$!1$&%)"!)$6%)-)'!"!1'-$#@)$I1+)%$I1'!$1% $ Z)3'1!)8$?!$+"-$'1!$1% '"##3$'!)',),$41%$!.)$+%!' $14$, !"#$61)!%3$1%$!.)$&%)"!1'$14$#!)%"%3$+1%@-$(0!$ ."-$"$ %)"!$61!)'!"#$"',$#)',-$!-)#4$+)##$!1$!.)-)$60%61-)-8 I%)Q$&"'$()$0-),$41%$4%))$(0!$!.)$6%)-)'!"!1'-$&%)"!),$1'$!.)$+)(-!)$"%)$"&&)--(#)$(3$"'31')8 $ $?!$-$ 61--(#)$!1$(03$!.-$-14!+"%)$:I%)Q$)-@!16;$+.&.$)'"(#)-$1')$!1$&1'!%1#$!.)$6%*"&3$-)!!' -$1'$!.)$ 60(#&"!1'-8$ I%)Q$6%1*,)-$2"'3$61--(#!)-$41%$!.)$'1'>!)&.'&"#$!1$6#"3$+!.$"$!)7!A-$6%)-)'!"!1'$"-$!$-$0-),$*)%3$ '!0!*)#3$"',$!.)$&%)"!1'$14$"'2"!1'-$-$."--#)>4%))8$ 9.)$!)7!$&"'$()$6#"&),$"'3+.)%)$1'$"$L,)-@!16ND5$'$"$20#!!0,)$14$61-!1'-$"',$-Q)-8$ J12)$)>+%!' $6#"3' $+!.$"'2"!),$!)7!$#@)$!.)$K10' $X")$S."' $X)"*3$?',0-!%)$[$"+1%@-$& '$"&!$"-$"'$ '-6%"!1'$'$!.)$6#"3$14$"$!)7!A-$6%)-)'!"!1'$:)*)'$4$!.)$+1%@-$14$KXSX?$"%)$2",)$'$M#"-.$"',$!.)%)$-$ )7!)'-*)$+1%@$1'$20-&$"',$!-$-3'&.%1'-"!1'Y,)>-3'&.%1'-"!1'$+!.$!.)$!)7!$+&.$-$'1!$61--(#)$+!.$ I%)Q;8 $$9.)$ %)-)"%&.$1' $!)7!0"#D $-06)%61-!1'$(3$ \2$]1-)'()% $+!.$.-$," %"2$ 61)2- $ -$"'1!.)%$ 61!)'!"#$-!20#0-$41%$&%)"!*)$+1%@$!."!$&10#,$()$,1')$1'$I%)Q8 $

2 ht/ tp:/ www.yhchang.com/DAKOTA.html 3 http://prezi.com/ 4 http://www.yhchang.com/ 5 http://www.well.com/user/jer/whatsnew.html

76 M1%$!.)$-!0,)'!-A$4%-!$+1%@-.16!5$"4!)%$-.1+' $-12)$-26#)$)7"26#)-$14$.1+$"$!)7!A-$6%)-)'!"!1'$&"'$()$ 6#"3),$+!.5$?$-0 )-!),$!."!$!.)$-!0,)'!-$&1'!'0)$"$X"@0$+.&.$+"-$"'$)7!%"&!$4%12$F0!Q$^"--2"''A-$ :"#"-$P'!1')$_1#1,');$(11@$,(-) ".$"/0- *15$0-' $I%)Q$!1$6%)-)'!$!.)$!)7!$'$"'$1% '"#$4"-.1'8$$?!$+"-$ "'$'!)%)-!' $)76)%2)'!$)*)'$4$21-!$14$!.)$-!0,)'!-$&.1-)$!1$0-)$!.)%$!2)$!1$+1%@$1'$*-0"#$,)- '$%"!.)%$ !."'$+%!' 8 ^)-,)-$()' $0-)%>4%)',#35$I%)Q$"#-1$."-$"'1!.)%$-!%)' !.$+.&.$-$'!%'-&$!1$!-$ ,>("-),$'"!0%)$14$ )7-!)'&)8$$9.-$)'"(#)-$,44)%)'!$0-)%-$!1$+1%@$1'$"$&1221'$I%)Q$&"'*"-$-20#!"')10-#35$"',$"#-1$!.)$%)"#> !2)$6%)-)'!"!1'$14$"$!)7!$'$!-$&%)"!1'8$$9.-$&1##"(1%"!*)$"-6)&!$&"'$()$*)%3$6%14!"(#)$,0%' $"$+%!' $ +1%@-.16H$"'$)'!%)$&#"--$&"'$&1'!%(0!)$!1$"',$'!)%"&!$+!.$!.)$+1%@-$14$!.)$1!.)%-$"!$!.)$-"2)$!2)8$ 9.)$6%1(#)2$-$!."!$!.)%)$"%)$-!##$-12)$:-!%"' );$(0 -$+.)'$!.)$'02()%$14$6)16#)$1'$"$!)7!$)7&)),-$4*) $ 6)16#)8 $ $ P#-15 $ !.) $ *-0"#Q"!1' $ -3-!)2 $ - $ '1! $ *)%3 $ -!%1' 5 $ (0! $ ! $ &"' $ () $ "' $ '!)%)-!' $ !11# $ 41%$ )76)%2)'!"!1'8 $ 9.)$-)&1',$&3&#)$LI"%&10%-$#(%)N$:M%))$]0';$+"-$*)%3$ )')%"#$"',$16)'8$?!$+"-$"$6%)-)'!"!1'$14$-12) $ , !"#$61)!%35$?'!)%"&!*)$M&!1'$"',$X36)%!)7!-$+1%@-$"',$!.)1%)-8 $

1.2 Interactive Fictions: $9.)$+1%@-.16$&1'--!),$14$&%)"!' $"'$?'!)%"&!*)$M&!1'$+!.$"$-6)&4&$6%1 %"2$&"##),$?'41%2 `$:_)%-1'$ a;8 L?'41%2$-$"$,)- '$-3-!)2$41%$'!)%"&!*)$4&!1'$("-),$1'$'"!0%"#$#"' 0" )8$?!$-$"$%",&"#$%)'*)'!1'$14$!.) $ +"3 $ '!)%"&!*) $ 4&!1' $ - $ ,)- '),5 $ 0,), $ (3 $ &1'!)261%"%3 $ +1%@ $ ' $ -)2"'!&- $ "', $ (3 $ !.) $ 6%"&!&"#$ )76)%)'&)$14$-12)$14$!.)$+1%#,b-$()-!>@'1+'$+%!)%-$14$?M8Nc ?'!)%"&!*)$4&!1'-$"%)$!)7!$+.)%)$!.)$%)",)%$&"'$2"@)$&.1&)-$"',$41##1+$,44)%)'!$6"!.-$"&&1%,' #38$?'$!.)$ '41%2->#@)$?'!)%"&!*)$4&!1'-$:!."!$"%)$14!)'$&"##),$?M$1%$G0-!$?'!)%"&!*)$M&!1'-$()&"0-)$!.)3$"%)$21%)$ 6160#"%;5$!.)$%)",)%$."-$!1$!36)$10!$.-$,)-%),$"&!1'-$1'$!.)$'!)%4"&)5$!1$",*"'&)$'$!.)$-!1%38$M1%$)7"26#)5$ 4$'$!.)$-!1%3$.)$+"@)-$06$'$"$%1125$.)$&"'$"-@$!1$L#11@N5$$!1$L16)'$!.)$,11%N5$!1$L 1$'1%!.N5$)!&8 9.)$4%-!$*)%-1'$14$?'41%2$+"-$,)*)#16),$'$TccU$(3$d%"."2$e)#-1'8

6 ht/ tp:/ prezi.com/mh2lgzcrhcwc/formes-figures-4-contrainte-decriture/ 7 Lutz Bassman, Les haïkus de prison, Verdier 8 ht tp://inform7.com/ 9 Ibid

77 P'$)7"26#)$14$?'!)%"&!*)$M&!1'H$2**)"(1."3*45$$(3$e&@$B1'!41%!Tf$1'$J6#"!!)%# .!

P'$'!)%6%)!)%$-$')),),$!1$%)",$"'$?'!)%"&!*)$M&!1'8$9.)$!+1$21-!$4"210-$?M$%)",)%-$"%)H J6#"!!)%# .!TT$41%$B"&8 d"% 13#)TV$41%$IS8 91$&%)"!)$"'$?'41%2$4#)5$-26#3$,1+'#1",$!.)$?'41%2$a$-0!TUC$!.)$-14!+"%)$-$"*"#"(#)$41%$"##$16)%"!' $ -3-!)2-$:F'075$B"&$"',$=',1+-;8$$P#!.10 .$!.)$?'41%2$),!1%$0-)-$"$-6)&4&$&1,)$!1$&%)"!)$?M$+.&.$."-$ !1$()$#)"%'!$()41%)$!.)$&%)"!1'$14$"$!)7!5$!.-$#"' 0" )$-$*)%3$)"-#3$"&/0%),$()&"0-)$!$-$-)$!1$'"!0%"# $ #"' 0" )$:. .$#)*)#;$"',$&"'$()$"$ 11,$'!%1,0&!1'$!1$6%1 %"22' 8 91$() '$"$!)7!5$"$%112$."-$!1$()$&%)"!),$+!.$!.)$-)'!)'&)$L9.)$g'"2)$14$!.)$%112h$-$"$%112N8$ $P $ ,)-&%6!1'$14$!.)$%112$&"'$()$",,),$"',$!$&"'$()$4##),$+!.$,44)%)'!$1(G)&!-$:&1'!"')%-5$40%'!0%)5$ 6)16#)888;8$$i!.)%$%112-$&"'$()$&%)"!),$"',$&1'')&!),$!1$!.)$1% '"#5$&)%!"'$&1',!1'-$1%$"$!2)%$&"'$() $ 261-), $ 8889.)%) $ "%) $ )',#)-- $ 61--(#!)- $ "##1+' $ " $ &126#)!) $ &0-!12Q"!1'5 $ "', $ 1') $ &"' $ )*)' $ ",,$ )7!)'-1'-T[$6%1*,' $')+$16!1'-$:41%$)7"26#)5$"'$W' #-.>M%)'&.$!%"'-#"!1'$6#0 '$41%$!.)$'!)%"&!1'-$ *1&"(0#"%3;8

10 ht/ tp:/ nickm.com/if/book_and_volume.html 11 http://ccxvii.net/spatterlight/ 12 http://ccxvii.net/gargoyle/ 13 http://inform7.com/download/ 14 http://inform7.com/write/extensions/

78 P$ (1*)5 !.)$?'41%2$a$'!)%4"&)8$$i'$!.)$#)4!5$!.)$+',1+$+.)%)$310$+%!)$!.)$&1,)5$1'$!.)$% .!5$!.)$+',1+$ +.)%)$310$&"'$%0'$+."!$."-$())'$+%!!)'5$%)",$!.)$,1&02)'!"!1'$1%$-))$!.)$-!%0&!0%)$14$310%$!)7!8 P$#1!$14$?'!)%"&!*)$M&!1'-$."*)$"#%)",3$())'$+%!!)'$+!.$?'41%2$"',$!.)%)$-$"$&1%'0&16"$14$)7&)##)'! $ %)4)%)'&)-$41%$-!0,)'!-5$-0&.$"-$!.)$!)7!-$^11@$"',$_1#02)$4%12$e&@$B1'!41%!Tj$1%$^",$B"&.')Tk$(3$"'$ J.1*!Q8 0%' $!.)$+1%@-.165$!.)$-!0,)'!-$+)%)$144)%),$!.)$&.1&)$()!+))'$!.%))$-0(G)&!-TaH$ i6)'$"$&.)-!H$!)##$"$(%)4$-!1%3$"(10!$"$&1'!"')%8$9.-$6"!.$-$"$ 11,$1661%!0'!3$!1$#)"%'$?'41%2$"- $ !.)$!)&.'&"#$)7 )'&)$-$#2!),$"',$-3'&.%1'-),$+!.$!.)$,)*)#162)'!$14$!.)$-!1%3#')H$&%)"!)$"$&1'!"')%5$ -)$!.-$&1'!"')%$"',$2",)$!$16)'"(#)$+!.$"'1!.)%$-6)&4&$1(G)&!5$0-)$!.)$L'-!)",N$-3-!)2888 !"#$I1)!%3H$&%)"!)$"'$?'!)%"&!*)$"(-!%"&!1'$+.)%)$%112-$"%)$'1!$%)"##3$6#"&)-$"',$+.)%)$'1%!.$- $ '1!$!.)$1661-!)$14$-10!.888$$?'$!.-$!.)2)5$!.)$-!0,)'!$&"'$)76#1!5$"-$1661-),$!1$()' $,-",*"'!" ),$(35 $ 61--(#)$(0 -$>$!.)3$&"'$6#"3$+!.$!.)$&1,)$"',$)76#1%)$!.)$(10',"%)-$14$!.)$6%1 %"28 e1$#2!-H$!.)$-!0,)'!-$&"'$0-)$?'41%2$'$!.)$2"'')%$!.)3$+"'!5$!1$&%)"!)$"'$?M8 P4!)%$"##$!.)$#)--1'-$'$!.)$-)2)-!)%$.",$)',),5$?$+"-$-0%6%-),$!1$,-&1*)%$!."!$!.)$+1%@-.16$+!.$?'41%2$ .",$())'$!.)$21-!$6160#"%$$:)*)%3$-!0,)'!$+.1$.",$"'-+)%),$!.)$)*"#0"!*)$/0)-!1''"%)$"!$!.)$)',$14$$!.)$ -)2)-!)%$*1!),$41%$?'41%2$ "-$!.)%$4"*10%!)$+1%@-.16;8$?'41%2$-$'1!$!.)$ 21-!$0-)%>4%)',#3$ -14!+"%)$ &126"%),$!1$!.1-)$6%"&!-),$'$1!.)%$+1%@-.16-5$(0!$!$-$,-!'&!*)$4%12$!.)$1!.)%$6%1 %"2-$+.&.$+)%)$ 4"2#"%$!1$!.)$-!0,)'!-$!.%10 .$!.)%$)76)%)'&)$+!.$(#1 -Y-1&"#$')!+1%@-8$ 9.)%)$"%)$1!.)%T`$6%1 %"2-$!."!$#"% )#3$%)-)2(#)$?'41%25$(0!$."*)$!.)%$1+'$-6)&"#!)-8$$P'$)7"26#)$-$!.)$ X0 1Tc$ 4%))+"%)$:+!.$!-$'!)%6%)!)%$X0 1%$"*"#"(#)$41%$B"&$"',$=',1+-;$ $+.&.$6)%2!$!.)$0-)$14$ 20#!2),"$:6&!0%)-$"',$-10',;8$e1!$#2!' $!.)$4&!1'$!1$"$60%)#3$!)7!0"#$,2)'-1'$&"'$()$!)26!' 5$(0!$ '$6%"&!&)5$!.)$!)7!0"#$41&0-$14$-14!+"%)$#@)$?'41%2$-$"$+"3$!1$@))6$!.)$"!!)'!1'$41&0-),$1'$!.)$&%)"!1'8$?!$ -$,44&0#!$41%$')+&12)%-$!1$,*,)$!.)%$)441%!$()!+))'$!.)$16!1'-$144)%),$(3$21,)%'$!)&.'1#1 3$"',$" $ #!)%"%3$'-6%"!1'$+.)'$0-' $')+$, !"#$!11#-8

15 ht/ tp:/ nickm.com/if/book_and_volume.html 16 http://collection.eliterature.org/1/works/shiovitz__bad_machine.html 17 http://prezi.com/pkcpzmw4fpvx/formes-figures-8-atelier-decriture-2/ 18 An other famous tools more often used before is TADS http://ifwiki.org/index.php/TADS_2 19 ht/ tp:/ www.generalcoffee.com/index_noflash.php?content=hugo&accessible=true

79 9$ 1 60(#-.$!.)%$+1%@-5$!.)$-!0,)'!-$&"'$0-)$"'$'!)%)-!' $6%1 %"2$!."!$'!)%6%)!-$"',$&1'*)%!-$?'41%2$a$ &1,)5$2"@' $!$,%)&!#3$0-"(#)$1'$!.)$+)(8$$9.)$+%!)%$14$"'$?'41%2$!)7!$G0-!$."-$!1$&163>6"-!)$.-$&1,)$1' $ !.)$I#"34&Vf$ 6#"!41%2$!1$2"@)$!$%)","(#)$1'#')$(3$)*)%3(1,38$9.)$#"-!$*)%-1'$14$I#"34&$'!) %"!)-$!.)$ ?'41%2$a$)7!)'-1'-8$$9.-$+"-$!.)$-)&1',$+1%@-.168$$$ 9.)$!.%,$&3&#)$+"-$41&0-),$1'$+1%@-$1'$!.)$6)%6.)%3$14$!)7!$:!.)$*,)1$ "2)-$41%$)7"26#);$"',$!.)$ +1%@-.16$'!%1,0&),$,"!"$*-0"#-"!1'$!11#-$"',$.1+$!.)3$&10#,$()$0-),$41%$&%)"!*)$+%!' 8$

1.3 Many Eyes and Data Visualisation $ e1+","3-$!$-$&1221'$!1$.)"%$"(10!$,"!"$"',$"$#1!$14$')+$-!0,)-$'!)%%1 "!)$.1+$+)$&"'$&%)"!)$!11#-$!1$ 6%1,0&)$')+$!36)-$14$'41%2"!1'8$$M1%$)7"26#)5$!.)$6.)'12)'1'$14$( $,"!"$60-.)-$0-$!1$!.'@$14$.1+$')+$ /0"#!"!*)$-!0,3$'!)%6%)!"!1'-$&"'$()$2",)8 l-' $"$#1!$14$,"!"$#)",-$!1$!.)$/0)-!1'$14$.1+$!1$%)",$!.)28$9.)$4)#,$14$,"!"$*-0"#-"!1'$."-$!1$"'-+)%$ !.-$/0)-!1'$"',$-'&)$#"-!$3)"%5$!$1&&06)-$"$ %1+' $6#"&)$'$!.)$-&)'&)-5$"',$)*)'$'$10%$6%*"!)$#4)8 =)##>,)- '),$*-0"#-"!1'-$: %"6.-5$!"(#)-5$)!&8;$."*)$"$61-!1'$14$"0!.1%!35$!.)3$ *)$!.)$26%)--1'$!."!$ !.)$'41%2"!1'$ )')%"!),$-$!%0)8$ $Z))6' $'$2',$!."!$!.)$")-!.)!&-$&"'$.)#6$!1$*"#,"!)$+."!$-$()' $ 6%)-)'!),5$*-0"#-"!1'$&"'$()$"'$'!)%)-!' $+"3$14$#3' 8$$=!.$!.-$6)%-6)&!*)$'$2',5$!.)$+1%@-.16$+"-$ &1',0&!),$"%10',$!.)$6#"3$+!.$&)%!"'$,"!"$*-0"#-"!1'-$!11#-$!1$ )')%"!)$4"#-)$ %"6.-5$2"6-$1%$," %"2-888$ "',$!1$+%!)8 B"'3$W3)-VT$-$"$+)($6#"!41%2$+.)%)$!.)$0-)%$&"'$-)',$!.)%$1+'$,"!"$"',$6%1,0&)$*-0"#-"!1'-$:+!.$!.)%$ ,"!"$(0!$"#-1$+!.$)*)%3$,"!"-)!$"*"#"(#)$1'$!.)$+)(-!);8$$9.)$6%1 %"2$-$,)&)6!*)#3$-26#)5$+.&.$&"'$ %)-0#!$'$6%1(#)2-$+!.$1(!"'' $,)"#$*-0"#-"!1'-5$2"@' $!$"$ 11,$&126%12-)$!1$+1%@$1'#3$+!.$!.)$ 6%)>,)4'),$,"!"-)!-8 P4!)%$!.)$*-0"#-"!1'$."-$())'$6%1,0&),5$!.)$0-)%$&"'$)761%!$!$+!.$"'$)2(),$&1,)$1'$)*)%3$+)(-!)$1%$ (#1 8$?$%)&122)',),$!."!$!.)$-!0,)'!-$&%)"!)$"$(#1 $1'$902(#%VV$41%$!.)%$+%!' 5$,0)$!1$!-$4"&#!3$14$0-)$ "',$!1$-0(-)/0)'!#3$261%!$!.)$*-0"#-"!1'-8

20 ht/ tp:/ playfic.com/ 21 http://www-958.ibm.com/software/data/cognos/manyeyes/ 22 https://www.tumblr.com/

80 0%' $!.)$+1%@-.16VU5$?$6%161-),$!.)$-!0,)'!-$!.%))$,)"-$41%$*-0"#-"!1'>&12('),$+%!' H !1$,)-&%()$"$4))#' $"',$!1$-.1+$.1+$!$)*1#*)-$1%$!1$&126"%)$,44)%)'!$'!)'-!)-$14$4))#' $"',$!1$ &122)'!$1'$!.)-)$-!"!)- !1$&%)"!)$"$4"#-)$L144&"#$6" )N$41%$"$-2"##$!1+'$%)#"!' $"$4&!1'"#$'&,)'! !1$+%!)$"$61)!&$'&126%).)'-(#)$!)7!$+.)%)$!.)$*-0"#-"!1'$&126#&"!)-$!.)$'41%2"!1'$'-!)",$ 14$&#"%43' $!8

1.4 Using tools to be more creative, be creative and liking to learn how to use tools

=!.$!.)$ 6%1 %)--1'$14$ !.)$#)--1'-5$ ?$+"-$-0%6%-),$!1$1(-)%*)$!.)$ -!0,)'!-A$ %1+' $4"-&'"!1'$"',$ "!!%"&!1'$41%$!.)$!)&.'&"#$!11#-5$"',$!.)%$+##' ')--$!1$#)"%'$.1+$!1$0-)$!.)2$41%$&%)"!*)$60%61-)-8$$ ?'$0-' $!.)-)$!11#-$41%$!.)%$1+'$6%1G)&!-5$!.)$-!0,)'!-$&%)"!),$!.)%$1+'$+"3$14$&1'!%1##' $!.)$!11#-$"',$(3 $ !%3' $!1$0-)$!.)$!11#-$'$"$b-0(*)%-*)b$2"'')%5$.",$!1$4%-!$#)"%'$.1+$!.)3$&1'*)'!1'"##3$40'&!1'),8 ?!$+"-$"'$)'%&.' $)76)%)'&)5$41%$2)$"',$?$.16)$41%$!.)$-!0,)'!$!118$^0!$!.-$)76)%2)'!$."-$60-.),$2)$!1 $ "-@$.1+$!$&10#,$()$61--(#)$!1$ 1$40%!.)%$"',$+.&.$!11#-$?$.",$!1$0-)$'$!.-$&"-)8 $ P$-.1%!$6%)-)'!"!1'$"',$&122)'!"%3$14$-12)$!11#-$!."!$&10#,$()$*)%3$0-)40#$!1$(%, )$!.)$ "6$()!+))'$!.)$ '1'>!)&.'&"#$"',$')+$!)&.'1#1 3$"-$+)##$"-$1(-)%*"!1'-$"',$-0 )-!1'-$41%$0-' $!.)2$"&&1%,' $!1$!.)%$ 6"%!&0#"%$&."%"&!)%-!&-8

2 News tools and perspectives: $

2.1 Create hypertext and IF: $ 9.)$,-!'&!1'$+)$2"@)$()!+))'$?'!)%"&!*)$M&!1'$"',$.36)%!)7!$-$!."!$!.)$%)",)%$."-$!1$+%!)$!1$,)-&%()$ !.)$"&!1'$.)$+"'!-$!1$"&&126#-.$'$?M$"',$.)$."-$!1$&#&@$1'$"$#'@$'$.36)%!)7!$!1$ 1$40%!.)%$'$!.)$!)7!8 ^0!$-12)!2)-$!$&"'$()$21%)$&126#&"!),$!1$-))$!.)$,44)%)'&)-$:+!.$-12)$?M$,)&1,)%-$41%$)7"26#);8$9.) $ !+1$&"!) 1%)-$()#1' $!1$!.)$'!)%"&!*)$!)7!8 ])&)'!#3$"$#1!$14$')+$!11#-$41%$%)",' $"',$&%)"!' $.36)%!)7!$."-$())'$,)*)#16),$"',$!.)$!11#-$41%$?M$ &1'!'0)$!1$()$26%1*),8

2.1.1 StorySpace: $ J!1%3J6"&)V[$ -$6%1("(#3$1')$14$!.)$21-!$4"210-$+%!' $6%1 %"2-$.-!1%&"##3$:"*"#"(#)$41%$B"&$"',$ =',1+-;8$?!$-$+!.$!.-$-14!+"%)$!."!$&)%!"'$%)'1+'$.36)%!)7!-$."*)$())'$6%1,0&),5$-0&.$"-"67!$01**18"(" !*09+:$(3$B&.")#$\13&)8 9.)$-14!+"%)$-$6%)-)'!),$"-$ m$"$.36)%!)7!$+%!' $)'*%1'2)'!$!."!$-$)-6)&"##3$+)##$-0!),$!1$#"% )5$ &126#)75$"',$&."##)' ' $.36)%!)7!-8$J!1%3-6"&)$41&0-)-$1'$!.)$6%1&)--$14$+%!' 5$2"@' $!$)"-3$"',$ 6#)"-"'!$!1$#'@5$%)*-)5$"',$%)1% "'Q)8$nVk 23 ht/ tp:/ prezi.com/iphxyw2hmbmv/formes-figures-seance-12/ 24 http://www.eastgate.com/storyspace/ 25 http://www.wwnorton.com/college/english/pmaf/hypertext/aft/index.html 26 http://www.eastgate.com/storyspace/

81 ?%!$-$'1!$4 ))5$'$4"&!$!A-$/0!)$)76)'-*)$:!$&1-!-$Vcj$,1##"%-;$"',$!$&"'$()$"$6%1(#)2$41%$0-' $!$'$"$&#"--5$ (0!$!.)%)$-$"$4%))$,)21$*)%-1'$$+.&.$"##1+-$!.)$!)-!$14$!-$40'&!1'"#!)-$"',$ *)-$!.)$-!0,)'!-$"$6%)*)+$ 14$+."!$&10#,$"$6%14)--1'"#$!11#$41%$+%!' $.36)%!)7!$&10#,$()8

2.2 Undum: $ ])&)'!#35$!.)%)$"%)$-12)$')+$6%1 %"2-$!."!$."*)$())'$,)*)#16),$"',$+.&.$"%)$4%))8$?"'$8$B##' !1'$."- $ ,)*)#16),$-12)$&1,)$41%$!.)$)"-3$ )')%"!1'$14$.36)%!)7!$1'$!.)$+)($$+!.$X9BFj8$9.)$-1#0!1'$-$&"##),$ l',02"#8 $ 9.) $ &1,) $ - $ ,1+'#1","(#) $ "', $ 0-)%>4% $ (0! $ !A- $ 6%)4)%"(#) $ !1 $ ."*) $ -12) $ ("-&- $ ' $ X9BF$ 6%1 %"22' 8

.!!6HYY+++8,)%,%"@"8&12Y!.)6#"3Y$ $ 9.)%)$-$"'$)7&)##)'!$!0!1%"#$&%)"!),$(3$?8$B##' !1'$!."!$-.1+-$!.)$61--(#!)-$+.&.$)76#"'$.1+$!.)$ 6%1 %"2$+1%@-$+!.$!.)$6%1 %"2$!-)#48"$ $ 9.)$6"%!&0#"%!3$14$!.)$.36)%!)7!-$&%)"!),$"%)$!.)$41##1+' H 27 ht/ tp:/ undum.com/ 28 http://undum.com/games/tutorial.en.html

82 • 9.)%)$-$"$-!"!0-$(17$+.)%)$!.)$-!"!)$14$2',5$-!0"!1'5$-@##$#)*)#5$21%"#$#)*)#<$14$&."%"&!)%-$&"'$()$ ,)-&%(),$"',$!.)$)#)2)'!-$14$!.-$-!"!0-$&"'$()$&."' ),$(3$!.)$"&!1'-$14$!.)$%)",)%-8$ • 9.)$.36)%!)7!$-$&1261-),$14$,44)%)'!$-)&!1'-$14$-2"##)%$!)7!-5$!.)$%)",)%$&"'$ 1$40%!.)%$'$!.)$-!1%3$(3$ &#&@' $1'$!.)$.36)%!)7!-5$!.)$!)7!$,1)-'A!$"#+"3-$,-"66)"%$"4!)%$"$,)&-1'5$1'#3$!.)$,)&-1'$*"'-.)-8$ 9.-$"##1+-$!.)$%)",)%$!1$%)",$"##$.-$!)7!$"!$!.)$)',$#@)$"$L'1%2"#N$!)7!$4$.)$+"'!-8 • 9.)%)$"%)$!+1$@',-$14$#'@-H$!.)$"&!1'$#'@-$"',$!.)$#'@-$!."!$2"@)-$310$",*"'&)$'$!.)$-!1%38$9.) $ "&!1'-$#'@-$&"'$&."' )$!.)$-!"!0-$14$!.)$&."%"&!)%-$1%$&."' )$-12)$14$!.)$!)7!$-)&!1'$+.)%)$!.)$%)",)%$-8$ 9.)$1!.)%-$#'@-$&."' )$!.)$!)7!$-)&!1'8 $ 9.)$2"'$6%1(#)2$41%$l',02$-$!."!$!.)%)$,1)-'A!$%)"##3$)7-!$"'$),!1%$41%$1')$!1$+%!)$,%)&!#3$+!.10!$ @'1+' $&1,)8$ 91$%)-1#*)$!.-$6%1(#)25$!-$&%)"!1%$."-$,)*)#16),$"$')+$-1#0!1'5$"$+)(-)%*&)$&"##),$_"%39"#)5$"#!.10 .$ l',02$&"'$()$"$ 11,$'!%1,0&!1'$!1$#)"%'$X9BF8 9.)$-!%)' !.$14$l',02$-$!."!$!$-$)"-3$!1$)761%!$"$&%)"!1'H$!$-$-26#3$&1261-),$14$-12)$.!2#$6" )-8 P!$!.)$-"2)$!2)5$!.)%)$"%)$'1$6%)>)-!"(#-.),$60(#&"!1'$!11#-H$310$."*)$!1$60(#-.$310%-)#4$1'$"$4!6$-)%*)% $ !.)$4#)-$4$310$+"'!$!1$-."%)$!.)$.36)%!)7!8 $

2.3 VaryTale $ _"%39"#)$-$"$6#"!41%2$+.)%)$310$&"'$%)",$"',$+%!)$.36)%!)7!-8$9.)$-!%0&!0%)$14$!.)$.36)%!)7!$-$'1!$!.)$ -"2)$"-$!."!$14$l',028$W"&.$.36)%!)7!$-$&"##),$"$L(11@N$"',$)"&.$(11@$-$&1261-),$14$2"'3$-0(>!)7!- $ &"##),$L-!1%3#)!-N8$K10$&"'$&.11-)$!.)$-!1%3#)!$310$%)",8$9.)$-!1%3#)!-$!."!$310$&.11-)$'4#0)'&)$!.)$-!1%3b-$ '"%%"!1'8

$ P$6"%!$14$2$$"4%12$W2#3$J.1%!$+.)%)$310$&"'$-)#)&!$"$-!1%3#)!8

83 P4!)%$-)#)&!' $!.)$-!1%3#)!5$310$&"'$%)",$!$"',$-12)!2)-$2"@)$"&!1'-$1%$"$4'"#$&.1&)8$.!!6HYY*"%3!"#)8&12Y%)",Y$ $ ?'-,)$!.)$-!1%3#)!-5$310$&"'$2"@)$"$&.1&)8$9.)$&.1&)$14$"$-!1%3#)!$-$"#%)",3$"'$"&!1'$"',$&"'$."*)$ &1'-)/0)'&)-8$=.)'$310$."*)$&122!$-12)$"&!1'8$K10$,-&1*)%$')+$-!1%3#)!-8 $

_"%39"#)$&"'$()$"$*"#0"(#)$.36)%!)7!>+%!' $!11#$()&"0-)$!A-$4%))5$0-)%>4%)',#3$"',$,1)-'A!$%)/0%)$ $" $ 6%1 %"22' $("&@ %10',8 X1+)*)%5$!$&"'$61-)$!+1$6%1(#)2-H P$!)7!$&"'$()$60(#-.),$"0!12"!&"##3$4$!.)$+%!)%$2"@)-$"$%)/0)-!5$(0!$."-$!1$()$*"#,"!),$(3$!.)$),!1%8$$^0!$ 4$"$-!0,)'!$+"'!-$!1$-."%)$.-$!)7!5$.)$&"'$,1+'#1",$.-$*)%-1'$"',$-)',$.-$4#)-$!1$.-$!)"&.)%8

84 9.)$2"'$6%1(#)2$-$!."!5$41%$-)&0%!3$%)"-1'-5$!.)$-3-!)2$!1$&%)"!)$"$+%!)%$"&&10'!$-$'1!$"0!12"!&5$-1$310$ ."*)$!1$"-@$41%$6)%2--1'$!1$-0(-&%()$!1$!.)$6#"!41%25$+.&.$&10#,$!"@)$-12)$!2)$:"#!.10 .$!.-$!2)$."-$ 6%1*)'$!1$()$*)%3$-.1%!$'$23$6)%-1'"#$)76)%)'&)5$"(10!$"$,"3;$(0!$!.-$-$'1!$"$&1'-,)%"!1'$!1$()$') #)&!),$ 4$310$+"'!$!1$6%1 %"2$"$#)--1'$+!.$!.-$!11#8 J12)$1!.)%-$!11#-$&"'$()$0-),$'$&#"--8$

2.4 Visual Novel

])'bI3Vc$41%$)7"26#)$:I3$41%$!.)$I3!.1'$&1,)$+!.$+.12$!$-$,)*)#16),;$-$"$4%))+"%)$41%$&%)"!' $*-0"#$ '1*)#-8$?!-$ %"6.&$*1&"(0#"%3$-$!"'!),$(3$2"' "$"',$G"6"')-)$,)- '$"'$:*-0"#$'1*)#-$"%)$+)##>@'1+'$'$ \"6"';8$X1+)*)%5$,)-6!)$-12)$&.#,-.$6&!0%)-5$!.)$6%1 %"2$&"'$()$0-),$!1$&%)"!)$&126#)7$.36)%!)7!-$+!.$ "$41&"#-"!1'$1'$!)7!$"',$6&!0%)-8$B0-&$&"'$()$",,),8$

P$&%)"!1'$14$"$-!0,)'!5$;4( #-1<"644"(3$"2)'$])2()%!

3 Other propositions 9.)$.36)%!)7!-$1&&063$"$*)%3$-6)&4&$"',$261%!"'!$6#"&)$'$, !"#$&%)"!*)$+%!' $"',$#)"%'' 5$(0!$!$&"'$ ()$&12('),$+!.$1!.)%$)#)2)'!-$"',$6%1 %"2-8$ $M1%$)7"26#)5$'02)%10-$61--(#!)-$14$')+$@',-$14$ '!)%"&!1'$+!.$Z')&!Uf$ "%)$4)"-(#)H$!.)$(1,3$14$!.)$0-)%$&"'$,%)&!#3$'!)%"&!$+!.$!.)$!)7!8$ $^0!$!.-$ &12('"!1'$14$,44)%)'!$!)&.'1#1 &"#$!11#-$%)/0%)-$"$&)%!"'$#)*)#$14$&1260!)%$-&)'&)$-@##-8$?$+##$'1+$ 6%)-)'!$-12)$1!.)%$!11#-$!."!$&"'$()$0-),$41%$'1'$!)&.'&"#$-!0,)'!-5$"#1' $+!.$(%)4$,)-&%6!1'-5$!."!$&"'$ -066#)2)'!$!.-$#-!$14$-0 )-!1'-8

3.1 3D and augmented reality 9.) S"*)$ +%!' $),!1%UT$ -$"$!11#$,)*)#166),$(3$^%1+'$l'*)%-!3$!1$-26#43$!.)$,)*)#16)2)'!$ 14$ "66#&"!1'-$41%$!-$SP_W$-3-!)2UV8$$9.)$SP_W$-$"'$U$)'*%1'')2)'!$+.)%)$310$&"'$6%1G)&!$*,)1-$"',$ !)7!$1'!1$410%$,44)%)'!$-&%))'-8$ $=!.$ 1 #)-$"',$61'!)%-5$310$"%)$"(#)$!1$'!)%"&!$+!.$"',$22)%-)$ 310%-)#4$'$!.)$6%1G)&!1'8$ 29 ht/ tp:/ www.renpy.org/ 30 ht/ tp:/ www.xbox.com/fr-FR/Kinect fDif erent API permit to use Kinect and to import embodiement interactions in personnal projects. 31 Downloadable at ht/ tps:/ wiki.brown.edu/confluence/display/wdm/Cave+Writing+Software+Installation 32 An article from wired: /http:/ www.wired.com/culture/lifestyle/news/2003/02/57334 , documentation is available on the Brown University website: ht/ tps:/ wiki.brown.edu/confluence/display/wdm/Cave+Writing+Resources

85 =! .$!.)$S"*)$+%!' $),!1%5$310$&"'$&%)"!)$"$!)7!$+.&.$+##$()$6#"3"(#)$'$!.)$SP_W$(0!$4$!$-$'1!$ 61--(#)$!1$."*)$"&&)--5$310$&"'$-20#"!)$!.)$%)-0#!$+!.$"$6%)*)+$-3-!)28$ $?!$&"'$()$"$ 11,$+"3$!1$'!%1,0&)$!.)$)#)2)'!$14$U$!1$-!0,)'!-$"',$!1$2"@)$!.)2$%)4#)&!$1'$.1+$!.)3$&"'$ 0-)$-6"&)$"',$21*)2)'!-8

P$-&%))'-.1!$14$!.)$),!1%

9.)$6%)*)+$14$"$-"26#)

3.2 Digital Poetry, playing social, annotating... one million other ways to write with new technology =%!' $+!.$')+$!)&.'1#1 3$&"'$()$*)%3$0-)40#$41%$-!0,)'!-$"',$41%!0'"!)#3$!.)%)$)7-!$23%",$61--(#!)-$ !1 $ )76#1%) $ ! $ ,44)%)'!#38 $ $91$ &126#)!) $ 23 $ 6%161-!1'-5 $ " $ #-! $ 14$ -12) $ 1!.)%- $ !11#- $ !1 $ 16)' $ 06 $ ')+ $ 6)%-6)&!*)-8$ ?!$+"-$'$Vffj$!."!$92$ib])##3$)76#"'-$!.)$=)($V UU$ "',$!.)%)$-$!.)$'&)--"'!$,"#3$%)6)!!1'$14$!.)$ -1&"(#!3$14$!.)$+)(U[5$'$!.-$&1'!)7!$.1+$&10#,$+)$41% )!$!.)$-1&"#$')!+1%@-$:M"&)(11@5$9+!!)%5$d11 #)$ o$G0-!$!1$&!)$!.%));$"',$'1!$!.'@$"(10!$+1%@' $+!.$!.)2$"',$2" '' $"$')+$+"3$14$+%!' D$

33 O'Reilly, 2005 34 Just some random works between so many: Chesbrough, Von Hippel...

86 S"'$!.)$-!0,)'!-$0-)$!.)%$-1&"#$')!+1%@-$!1$&%)"!)$"$4&!1'D$1$!.)3$')),$!1$&%)"!)$"$6-)0,1'32$"&&10'!$!1$ 6%1,0&)$-12)!.' $')+D$=."!$-$!.)$61'!$()!+))'$-1&"#$+%!' $"',$&1##"(1%"!*)$+%!' $:+."!$,44)%)'&)$ +.)'$!.)3$+%!)$1'$"$+@$"',$1'$"$-1&"#$')!+1%@D;8 9.-$4)#,$&"'$()$*)%3$6%14!"(#)$!1$)76#1%)5$)-6)&"##3$()&"0-)$!.-$&10#,$-)'-!Q)$!.)$-!0,)'!-$!1$!.)$6%*"&3$ %0#)-$"',$!1$()!!)%$0',)%-!"',$+."!$"',$.1+$!.)3$&"'$-.1+$+."!$!.)3$+"'!$!18 ?!$-$61--(#)$!1$L-0(*)%!N$!11#-$!1$6%1,0&)$'"%%"!1'$1%$61)!%35$2',$2"6$-14!+"%)- Uj$&"'$()$"$ 11,$.)#6$41%$ !.)$&%)"!1'$14$"$.36)%!)7!$+!.$"'1!.)%$-14!+"%)5$(0!$!$&10#,$()$,%)&!#3$0-),$!1$+%!)$'$"$*)%3$"%(1%)-&)'!$ +"38 =4( #>?$-$"$+)##>@'1+'$-14!+"%)$41%$, !"#$+%!' $+!.$+.&.$!$-$)"-3$!1$&%)"!)$"'2"!),$!)7!8$P''1!"!1'-$ -3-!)2$#@)$@-<1$".$"!$5/ >A"&10#,$()$"$ 11,$-10%&)$14$'-6%"!1'8

$ J!0,)'!-$+"'!$!1$#)"%'$!1$0-)$!11#-$!."!$&"'$.)#6$!.)2$'$!.)%$40!0%)$6%14)--1'"#$#4)8$e1!$"##$14$!.)2$."*)$ !.)$"2(!1'$!1$()&12)$!)&.'&"#$)76)%!-5$!.)3$G0-!$+"'!$!1$()$"(#)$!1$"&&126#-.$!.)%$!"-@-$+!.10!$ -044)%' $!11$20&.8$^3$ *' $!.)2$!.)$,)-%)$!1$6#"3$+!.$!)&.'1#1 &"#$!11#-5$!.)$()')4!-$"%)$20#!6#)H 9.)3$#)"%'$.1+$!1$0-)$-14!+"%) 9.)3$"%)$'!"!),$'!1$!.)$--0)-$14$')+$!)&.'1#1 3$"',$&"'$)*"#0"!)$-12)$!)&.'&"#$"%!)4"&!- 9.)3$"%)$-)'-!Q),$!1+"%,-$, !"#$+%!' $"',$21%)$ #1("##3$-6)"@' 5$!1+"%,-$!.)$:61-!;, !"#$"%!$ 6%1,0&!1' 9)"&.)%- $ "', $ "%!-!- $ ')), $ !1 $ ()$ &%)"!*) $ !1 $ '*)'! $ ')+ $ +"3- $ !1 $ 0-) $ !11#-$ "', $ !1 $ 6%161-) $ -12) $ ')+$ 6)%-6)&!*)-$41%$''1*"!1'8$$=!.$!.)$, !"#$1661%!0'!)-5$"$')+$6#"3 %10',$)7-!-$+.)%)$)*)%3(1,3$&"'$ ."*)$)'10 .$-"',$!1$(0#,$')+$&"-!#)-8

35 ht tp://freemind.sourceforge.net/wiki/index.php/Main_Page : FreeMind is a free mind mapping software 36 ht/ tp:/ www.adobe.com/fr/products/flashplayer.html 37 http://www.iri.centrepompidou.fr/outils/lignes-de-temps/

87 Al l the links have been verified the 18th June 2012

562%#-/! [1] BASSMANN, Lutz. Les Haïkus de prison, Verdier 2008 [2] O'REILLY, Tim. What is web 2.0. 2005 ht/ tp:/ oreilly.com/web2/archive/what-is-web-20.html [3] CHESBROUGH H. W., VANHAVERBEKE W., WEST J. Open innovation : researching a new paradigm. Oxford : Oxford University Press, 2006. g[h$X?IIWF$W8$_ie8$'$5*B0(!-C-1<"-11*D(!-*18$S"2(%, )5$B"--8$H$B?9$I%)--5$Vffj8

List of E-writing: [1] Young Hae Chang Heavy Industrie : ht/ tp:/ www.yhchang.com/ [2] Jim Rosenberg: ht/ tp:/ www.well.com/user/jer/whatsnew.html [3] Nick Montfort, Book and Volume ht/ tp:/ nickm.com/if/book_and_volume.html [4] Dan Shiovitz, Bad Machine : ht/ tp:/ collection.eliterature.org/1/works/shiovitz__bad_machine.html [5] Michael Joyce, Afternoon a story (web version) ht/ tp:/ www.wwnorton.com/college/english/pmaf/hypertext/aft/index.html [6] Emily Short, Bee ht/ tp:/ varytale.com/read/ [7] )%,%"@"H$Y.!!6HY +++8,)%,%"@"8&12Y!.)6#"3Y$

List of programs: gTh$I%)Q$Y.!!6HY 6%)Q8&12Y$ gVh ?'41%2$aH$.! !6HYY'41%2a8&12Y$ gUh$J6#"!!)%# .!H$Y.!!6HY &&7*8')!Y-6"!!)%# .!Y$ g[h$d"% 13#)H$.!Y !6HY &&7*8')!Y "% 13#)Y$ gjh$X0 1H$$.!Y !6HY +++8 )')%"#&144))8&12Y',)7p'14#"-.86.6D&1'!)'!q.0 1r"&&)--(#)q!%0)$$ gkh$I#"3M&$H$$.!Y !6HY 6#"34&8&12Y$ gah$B"'3$W3)-$H$.!Y !6HY +++>cj`8(28&12Y-14!+"%)Y,"!"Y&1 '1-Y2"'3)3)-Y$ gch$902(#%$H$YY.!!6-H +++8!02(#%8&12Y$$ gTfh$J!1%3J6"&)$H$.!Y !6HY +++8)"-! "!)8&12Y-!1%3-6"&)Y$ gTTh$l',02$H$.!Y !6HY 0',028&12Y gTVh$])'63H$.!Y !6HY +++8%)'6381% Y$ gTUh$S"*)$+%!' $),!1%$H$ .!YY !6-H +@8(%1+'8),0Y&1'4#0)'&)Y,-6#"3Y+,2YS"*)o=%!' oJ14!+"%)o?'-!"##"!1'$ gT[h$M%))B',$H$$.!Y !6HY 4%))2',8-10%&)41% )8')!Y+@Y',)786.6YB"'pI" ) gTjh$F ')$,)$!)26-H$$Y.!!6HY +++8%8&)'!%)6126,1084%Y10!#-Y# ')->,)>!)26-Y$ gTkh$P,1()$M#"-.$H$.!Y !6HY +++8",1()8&12Y4%Y6%1,0&!-Y4#"-.6#"3)%8.!2#$

7$&2)/ gTh$PJJiS?P9?ie$]WMFW9$W$FW99]WJ8$@-!!E0(!0$"15E0-G$"$!"B($!$0(8$I"%-$H$e1s--5$Vffk8 [2] BÉLISLE C. Lire dans un monde numérique état de l’art. Villeurbanne : Presses de l’enssib, 2011. [3] BOOTZ P. Alire la revue de littérature animée et interactive, Le salon de lecture électronique : 1989- 1995. 1995,.

88 [U4] BO CHARDON S. Un laboratoire de littératures : littérature numérique et Internet. Paris : Bibliothèque publique d’information, Centre Pompidou, 2005. [5] BOUCHARDON S. Littérature numérique : le récit interactif. Paris : Hermès science publications : Lavoisier, 2009. [6] HAYLES N. K. Electronic literature : new horizons for the literary. Notre Dame, Ind. : University of Notre Dame, 2008. [7] HAYLES N. K. How we became posthuman : virtual bodies in cybernetics, literature, and informatics. Chicago, Ill. : University of Chicago Press, 1999. [8] HAYLES N. K. My mother was a computer : digital subjects and literary texts. Chicago : University of Chicago Press, 2005. [9] HAYLES N. K. Writing machines. Cambridge, Mass. : MIT Press, 2002. [10] LINARÈS S. Figures de la narration. Caen : Lettres modernes Minard, 2010. [11] MONTFORT N. Twisty little passages : an approach to interactive fiction. Cambridge, Mass. : MIT Press, 2003. [12] SAEMMER A., MAZA M. « Alexandra Saemmer, Monique Maza présentent E-formes : écritures visuelles sur supports numériques. ». Saint-Étienne : Publ. de l’Univ. de Saint-Étienne, 2008a. [13] SAEMMER A. Matières textuelles sur support numérique. Saint-Étienne : Publications de l’Université de Saint-Étienne, 2007. [14] ZREIK K. Apprentissage par l’interaction. Paris : Europia production, 1997. [15] ZREIK K., TROUSSE B. Organisation de la conception. Paris : Europia, 1994. gTkh$u]W?Z$Z8$6//0$1!- (<$"#*55$H5(B#-1$"I",5(1H5(B#-1$"4$(01-1

Disrupting 3D models

Everardo REYES-GARCIA LabSic, Université Paris 13 99, Jean-Baptiste Clément Villetaneuse, France, 93430 [email protected]

Abstract renders it. For us, the importing process implies to Media art is constantly looking for innovative ways read its the STL original geometry and to present it to produce objects and processes and to as an ordered path of points or triangles. communicate with/through them. In this work we Processing supports lines and quads as well, but introduce an experimental way to view and interact we did not retain them because of an aesthetic with digital 3D models. We have developed a choice. Figure 1 shows an imported model by simple application that allows users to manipulate, Bernard Dejardin from the open repository decompose, and recompose models. We show scifi3d.com. examples of three type of models: realistic, generative, and what we call “motion structures”. We discuss our results from the standpoint of computational aesthetics, visual semiotics and software studies. Keywords media art, aesthetic computing, visual semiotics, software studies.

1 Introduction 3D models are generally considered as digital Figure 1 objects made of geometry that simulate width, height, and depth inside a computational Once a file has been loaded, the user is able to environment. Although their origins can be traced interact inside the 3D view environment. A series to computer graphics during the 60s, when of GUI sliders indicate modifications upon three pioneers created computer simulations, virtual factors: factX, factY, and factZ. Additional buttons reality, virtual worlds, and visual effects in film, apply transformations, change mode view (points nowadays we encounter them in graphic design, or triangles), and reset the model to its original video games, architecture, art, advertising, Web, state. industrial design, scientific visualizations, and digital humanities. The idea of the system is to invite users to perform transformations on the models. The In this contribution we introduce an resulting images from transformations are basically experimental way to view and interact with 3D abstract. Some would be appealing but disrupting models. We also present ongoing work to produce at the same time. They show how the same 3D models, based on images sequences. Our aim is quantity of triangles in a model may be to contribute to aesthetic viewpoints and recomposed. Of course because STL relies heavily explorations. The results obtained are then studied on tessellation, triangles are never separated. That from the angle of visual semiotics and software means, geometry is not spread they rather create studies. new solid forms. If a transformation is made on the view mode of POINTS, then the result looks like a 2 Disrupting models particle explosion. Figure 2 shows two transformations from the colored triangles mode, We have developed a simple application that obtained with different factor values. allows users to interact with 3D models. Technically speaking, we used Processing with ModelBuilder and ControlP5 libraries. The application basically takes a STL model and

91 Figure 2 Figure 3: Three views of Anonymous mask

But what can we say about these new forms? Are 4 Visuality they a kind of computational aesthetics? Can they teach us something from the visual semiotics From the perspective of visual studies there are standpoint? Or what about software studies? some points that can be observed. One of the main contributions of visual semiotics is to study visual 3 Aesthetic provocations phenomena by taking into account both figurative and plastic elements. That means that the interest is These images can be approached from the not only on recognizable figures and symbolic computational aesthetic standpoint. As it has been iconography but also on the material properties of recalled by Zhang, Harrell, and Ji, computational images. In 1992, the Groupe Mu identified three aesthetics aims at answering “How can the main plastic elements: colors, forms, and textures. computer automatically generate various forms of The work of American philosopher Charles visually aesthetic expressions?” The generation of Sanders Peirce is also valuable. He distinguished forms requires to consider computational categories of signs according to three relationships: complexity, i.e. how much of the artistic decision the sign itself, the object, and the intepretant. In the is delegated to the machine power. present article we will concentrate on the Groupe Mu perspective. Our intention is to provoke aesthetic experiences by viewing differently a 3D model. Our strategy is We believe the importance of focusing on to decompose an iconic image and to convert it material visual properties is to develop a more into indexical and unrecognizable forms. We go rigorous visual literacy. This endeavor requires the from realism to abstractionism; from order to eye to look sharper at visual material stimuli but at random disorder. the same time requires the brain to create and connect abstract relationships of meaning. To For a user to obtain results with our system, she further develop our point, allow us to discuss must negotiate with the graphical interface. The briefly some theoretical references. user experience model is cyclic: decomposing an object and then recomposing it to be decomposed Danish researcher Louis Hjelmselv elaborated again. She must deal with parameters in order to his theory of glosemiotics based on the distinction generate a compelling shape. An outcome involves of two planes in front of a signifying sign: then to focus both on the input model and the expression and content. Both concepts could be artistic decisions of the system. associated to what Swiss linguist Ferdinand de Saussure called signifier and signified, To discuss another example, allow us to respectively. But for each plane there are two more remember the Anonymous mask which is now levels: the form of expression and the substance of heavily associated with political and cultural expression, and accordingly form of content and movements. A STL model has being distributed in substance of content. On one hand, the form is P2P sites for users to download it and 3D printing related to a constructed and abstract meaning, a it. With the growing introduction of rapid sign, or what structuralists called “a structure”. On prototyping in art fields and the relatively easy the other hand, the substance is the material part, it access to 3D printers, we may ask: are generative is articulated by the form in order to signify. and abstract forms inviting to produce a physical Hjelmselv postulates that a form can have various object as well? substances (eg. graphical, gestural, phonetical), but not the opposite.

92 As it can be seen, the glosemiotic model the disruptions when it is decomposed. A common provides a more detailed way to approach paradigm of industrial 3D modeling goes from signifying processes than Saussure’s. There are simple to complex. In our system, we do the always two levels to consider, whether we opposite. We apply the basic transformations of concentrate on the material or on the content. If we move-rotate-scale in stochastic and chaotic apply the model to plastic signs, we can talk about fashion. Hence, every transformation, even with plastic signifiers and plastic forms. French similar parameters, is likely to generate different semiotician Algirdas Greimas has already made an structures. Abstract as they are, they may foster effort on understanding abstract and culturally readings and meaning effects of: provoking visual productions. One of the main roundness/harshness, flat/jagged, union/dispersion, questions is of course: what and how these small/tall, stretched/squashed. For a user, the only productions mean? reference to a figurative reading is through the original model. The structural analysis offers a logic and ordered methodology for a first approach. Given a visual But is it possible to say that these meaning production inside a 2D space –a painting, a poster, effects are homological? Is there a universal or an a computer screen- we can observe topologic ideal shape of harshness? Is it useful to categorize categories that allow its segmentation into smaller plastic formants? What happens if there is no parts: top/bottom, left/right, periphery/center, etc. figurative background as reference? Although the Inside the topologic scheme we can articulate scope of this paper is not to answer these signifier units according to two categories: questions, we clearly identify the need to continue chromatic and eidetic. The first is related to hue, experimenting with abstract digital objects. saturation, and brightness. The second is related to Moreover, we find it relevant to explore them forms and its role is to establish the discretion of within a simulated 3D space. The fact of zooming different signifier units. The combination of both and orbiting is required for the visual experience. categories gives “plastic formants”: particular arrangements of signifiers that constitute a sign. Figure 4 shows an imported object generated with Structure Synth and a transformation. Once we have plastic formants we can perform Generative art has reputation of producing plastic readings. In linguistics, the syntactical complex and non-iconic images and structures. organization of forms allows to treat them as This is just an exercise to think about abstract semiotic processes. The reading is done on two models as input and abstract structures as output. axes. The paradigmatic axis defines relationships according to the couple “this or that”. It allows to recognize the presence in opposition to absence: e.g. this painter’s palette of colors and not other. The syntagmatic axis is constructed from relationships “this and that”. It informs us on the mode of co-presence of plastic formants. For example, plastic contrast operates by the co- presence of opposite formants from the same plastic category. Figure 4: Transformation of an object generated Greimas postulates the principle of with Structure Synth homologation between plastic categories and content categories. For example ,the opposition 5 Motion structures top/bottom with euphoria/dysphoria. For him, it is not important to know if such homologations “Motion structures” is a work in progress that represent cultural conventions or universals, it is aims at creating and studying 3D models from the principle itself of modus operandi that matters. image sequences. Our first explorations have been Of course, the homological categories come from made on film sequences. From the perspective of the form and not from the substance. film studies, our contribution is an effort to start thinking about moving pictures as objects and to As we said, in our work we are interested on analyze them through techniques and processes provoking different readings. The fact of loading from other disciplines. Figure 5 shows the motion an iconic 3D model makes it easier to understand structure of a shot from the bending-Paris sequence

93 in the film Inception (Christopher Nolan, 2010). on the input kind of model. The process to obtain this object involved the use of Quicktime Pro, Automator, ImageJ, Blender, Before developing our application, we asked and MeshLab. ourselves is it was possible to produce our generative transformations with existing software. Although Maya and Blender, among others, allow to enhance their functionalities with scripting code, we decided to create our own system mainly because we wanted to deploy a simple user interface to interact with objects. Indeed, Maya and Blender are complex environments that cover topics from modeling to animation and rendering. Furthermore, they handle their own file formats. It is possible to export/import to other formats but the Figure 5: Right: Motion structure in Fiji. Left: main workspace relies on their own. For us, we use the structure imported in our system STL format, which is widely adopted for 3D printing and would satisfy our previous question Basically, our visual productions put in evidence on printing a complex and disordered model. But the shape of movements. Instead of analyzing perhaps the main reason behind the production of a camera movements or character performances, we system is the cultural practices that it supports. A trace the transformations within the image itself. complex and integrated environment is suited for This allows us to explore the form of indexical many tasks required by cultural industries but not signs of moving pictures, to construct shapes, and precisely for a particular artistic approach and to explore them in 3D fashion. research need.

We believe that motion structures are also a kind Currently, we observe an increasing of abstract and disrupting models. Our initiative communication between 3D software to support toward film may be considered as iconoclast, in and exchange file formats. For example, Collada terms of Boris Groys, but it is also a kind of DAE is supported in Maya and Blender. In another transcoding. An animated sequence is transformed example, a structure generated in Structure Synth into an image sequence, that is a series of image may be exported as OBJ, which in turn can be files. Then, these files are digitally processed to get exported as Collada DAE or STL in Blender. But stacked and removed background. Finally, they are there are still some missing pieces if we want to exported as a 3D object, which we require to experiment with remixability of media species. remesh to obtain a low-res version that can be Imagine that you could record a video right from a loaded in our system. The final outcome may not Web browser. This would be useful for recording resemble the high-res and neat image of film, but it navigation sessions, videoconferences, or for does preserves fundamental plastic signs. studying in detail navigation patterns. Consider now that you could export as OBJ a visualization When a motion structure is the input for made with the plug-in Tilt in Firefox. It seems a transformations in our system, the output renders a large quantity of media productions are adapted for highly complex disordered geometry. This is due to Web but not the contrary. Consider the exportation the fact that a motion structure has an average of of PSD or AI to PNG or JPG, AE to MOV or AVI. 32,000 faces, which is almost the threshold Yet, today we encounter new objects that in most supported by our system. While an optimized cases are reduced to image or video for Web model would have 2,000 faces, we should also note distribution: media and network visualizations, that the original OBJ obtained from Fiji has more maps, tags, augmented reality, etc. than 500,000 vertices and more than 1 million faces. Innovative uses of software foster at the same time a culture of transcoding. We believe it is 6 Thoughts on software important to document traces of media in order to theorize in the long tail. As we mentioned earlier, we used Processing to develop our viewing/transformation system. 7 Conclusion Regarding the acquisition of 3D models, we used several different software applications depending In this article we have introduced a novel and

94 experimental way to view and interact with 3D models. The main idea is to provoke aesthetic experiences with digital objects. We developed a simple application using Processing that allows to load, view, explore, decompose, and recompose a 3D model. To perform a transformation on an object means to parameterize a GUI, that behind the scenes moves, rotates, and scales stochastically and chaotically the fundamental geometry. The new structures would be appealing and disrupting at the same time. From the standpoint of visual semiotics these processes may foster different kinds of lecture, ranging from iconic to indexical, from figurative to plastic elements. From the point of view of software studies, we argue that more efforts are needed to explore the transcoding of digital media species. To adopt media art as experimental and artistic practice means also to challenge current paradigms and supported tasks of our current tools.

8 Acknowledgements I am thankful to Marius Watz, who gave me support with some parts of the programming code and with implementing this idea using his library Modelbuilder. References [1] Fiji. http://fiji.sc [2] A. Greimas, 1984. Sémiotique figurative et sémiotique plastique. Actes sémiotiques.VI, 60. Paris: CNRS. [3] Groupe Mu. 1992. Traité du signe visuel. Seuil, Paris. [4] B. Groys. 2008. Thinking In Loop: Three Videos on Iconoclasm, Ritual and Immortality. DVD. ZKM, Karlsruhe. [5] L. Hjelmslev. 1968. Prolégomènes à une théorie du langage. Minuit, Paris. [6] L. Manovich. 2008. Software takes command. UCSD, San Diego. [7] Meshlab. http://meshlab.sourceforge.net/ [8] Modelbuilder. http://workshop.evolutionzone.com/ [9] Structure Synth. http://structuresynth.sourceforge.net/ [10] Tilt. https://addons.mozilla.org/en- US/firefox/addon/tilt/ [11] K. Zhang, S. Harrell and X. Ji. 2012. Computational Aesthetics: On the Complexity of Computer- Generated Paintings. In Leonardo, 45/3. MIT Press, Cambridge, MA.

Digital Preservation of Ju Ming Stone Sculpture in Taiwan

- the documentation and regeneraition of arts

Naai-Jung SHIH, Chia-Yu LEE Queenie LIN, Ruo-Xuan CHEN Department of Architecture Conservation Center, Juming Museum National Taiwan Univ. of Science and Tech. No. 2, Xishihu, Jinshan Dist. 43, Section 4, Keelung road New Taipei City, 20842, Taiwan, Taipei, 106, Taiwan, R.O.C. R.O.C. [email protected] [email protected] [email protected] [email protected]

Luca et al., 2011; Duran and Aydar, 2112; Abstract Remondino, 2011). The results must also be 3D digital preservation of arts was conducted presentable for large audiences for pedagogy on the sculptures created by famous Taiwanese purposes in the virtual world (Trapp et al., 2012). sculptor, Ju Ming, which is a one year project to The public needs to access cultural heritage scan 9 copper and 55 sand stone sculptures. subjects over the Web in real time interaction Since the stone sculptures are currently (Hernandez, 2008; Manferdini and Remondino, deteriorated, the intended pace of one per season 2012). In contrast to the appearance of some for 3D digitization becomes very urgent; as both models generated in a lab with lighting controls the shape and appearance must be retrieved (Sitnik et al., 2012), this project was conducted before further weathering damages occur. A outdoors, where daylight changes were a great hand-held device, the Artec 3D color scanner concern. HM-T, was used as the main capture tool for 2 Map of collaboration stone sculptures, while the Cyrax 3000 long- range laser scanner was used for the larger The sand stones were collected from local areas, copper sculptures. and due to their relatively low strength, layers of skins were separated from the surface or 1 Introduction deteriorated into pieces. The Museum Digital preservation efforts were made on stone commissioned the Botany Department, Taiwan sculptures made by a famous Taiwanese sculptor, Natural Museum of Natural Science, to conduct Ju Ming. The one year project is to scan 55 sand relevant tests in 2008, which determined that micro stone sculptures in 3D (Fig. 1), which are currently organic hyphae can indeed grow along the cracks deteriorated, at a pace of one per season. The 3D and pores of the lithic material, thereby causing digitization becomes very urgent and seeks to damage to the artworks. From the perspective of salvage both shape and appearance before further preventive conservation, considerations must be weathering damages occur. given in future conservation works to how mosses, Preservation is an integral task of the planning, lichens, etc. are causing biological deterioration to designing, and resources, and involves many roles stone sculptures, as well as how to avoid physical from data acquisition to web page creation (Haval, damage that may arise from frequent surface 2000), and all maintenance efforts should be cleaning. The study focuses on how to strike a planned ahead (Guarnieri et al., 2010). Information balance between sufficient surface cleaning and the should be in the simplest possible digital format to reduction of biological growths on stone sculptures. minimize the requirements for specific or possibly The collaborative efforts are listed in chronological obsolete software (Hedstrom, 1998). order. Beyond the three-dimensional data, a large  In 2008: Professor Lin C.K., Botany amount of heterogeneous data is collected during Department, Taiwan Natural Museum of the analysis of buildings. Integrating Natural Science: Tests on mosses and lichens. heterogeneous data requires the management of the Helen Jacobsen (stone conservator from collected information regarding measurements, Germany): Condition survey of the artworks analysis, and interpretation of building shapes, and and tests for restoration. structural inspections (Andres et al., 2012; De  Between 2009-2010: Tests and studies by 97 stone conservator Helen Jacobsen for the to identify the configuration of a sculpture (Fig. 3). restoration work of the stone sculpture While the as-built digital models represent textures display area. Analyses conducted on the and the actual perception of size and proportion, sandstone’s salt content, sorption isotherms, the scan data enable the monitoring of detailed and hygric dilatation. configurations changes of colors, shapes, and  In 2011: The Department of Transmission biological deteriorations. The conservators now and Learning, Headquarters Administration can refer structure features to 3D models for future of Cultural Heritage, the Council for Cultural conservation references. Affairs, Executive Yuan: Documentation for digital preservation, field surveys of artwork 4 Web 3D environment deterioration, and restoration proposals. A Sculptures data also serves public education and team led by Professor Shih N.J., Department promotional purposes. Interactions between data of Architecture, National Taiwan University and users are conducted through the Internet (Fig. of Science and Technology: 3D digital 4) with 3D graphic plug-ins, such as Otaga, collection, preservation, and value-added Cortona, or Octree, for easy access and minimum applications project for Ju Ming’s Sculptures. maintenance efforts. Instead of a specific viewing Deterioration of stone sculptures mainly arises format or virtual environment, 3D models are from natural weathering and human damage. presented in the Flash format. As the models are Works placed in the outdoors are susceptible to distributed, the copy right of sculpture-related data expansions /contractions (due to temperature can be maintained without compromising visual differences), erosion by rain and wind, organism- quality or demanding higher computer graphic induced effects, and the deposition of various standards. materials from the atmosphere, all of which can Color-textured rapid models (Fig. 5) are printed increase weathering effects. At the same time, by a rapid prototyping (RP) machine in a smaller human damage to the stone can easily occur from scale as an alternative to presenting the objects to people touching, climbing, or leaning on the the public. Digital and scaled physical models are sculptures. used as reference in order to avoid any possible Weathering has a huge impact on sculptural future weathering damages. works. It is the gradual fragmentation and decomposition of stone due to exposure to air, 5 Data integration and interaction design water, and organic action. The three types of weathering - i.e., mechanical (Fig. 2), chemical, This project attempts to integrate resources, data and biological weathering, often occur design, and planning with existing working concurrently and affect each another. processes in preservation. Scanned data was formerly designed using the web as the only 3 Issues and tools interface with the public or conservators, while this project divides the user interface into two parts: This challenge is twofold, namely, texture and interactive web pages for the public and specific details. Sculpture appearances constantly change 3D applications for the conservators. A typical during the day from dawn to dusk, or between example is the measurement function, which clear sky, overcast, and rainy days. If a single conservators use to inspect the model in detail. sculpture is not completely scanned, the subject Other functions are also applicable, such as must be re-scanned for minor differences that may viewing the model in mesh mode for geometric exist in terms of brightness, saturation, contrast, or inspections. Most important of all, the working gamma. Modifications may have to be made to models are for interior uses with the resolution of separate scans. Since there were always regions or polygons reaching 100000 or higher. orientations missed in the first scan, following-up In order to make the best of all scans, the data scans were usually conducted in 3-4 additional are designed to be smoothly transferred from the trips. The number of scans varies from 20 to over project to the museum conservators by actually one hundred, depending on the size and complexity allowing them use the data in all possible ways. of a sculpture. The scan data were delivered to the owner in Photogrammetry bears limits in modeling advance, with the introduction of 3D software to multilayer, overlaid, and concaved crack bridge the process between traditional work flow conditions, thus, a hand-held 3D short range and that with digital data. As hands-on digital scanner is more feasible. A hand-held device, the experiences are encouraged, the conservators can Artec 3D color scanner HM-T, is used as the main perform simple data maintenance and simplify capture tool. The scans contribute to the inspection future life cycle updates. procedures of the museum conservators by helping 98  Hands-on for all: The experience for all those conservators. Future research includes the life- involved especially original conservators. cycle monitoring of weathering data based on The purpose is to encourage them to conduct comparisons of the virtual models and the everyday works based on the digital models, sculptures. such as converting formats, measuring distance, changing resolution, or comparing 7 Acknowledgements historical scan data to actual physical objects. This project is sponsored by the NPO Juming  Maintained by all: By properly planning Culture & Education Foundation under university digital modeling and related data, anyone can project number 0205. The author wishes to show update the data or trail contents, as follows. their appreciation.  Contents of pop-up windows: simply editing, replacing images, adding text, References or promotions to images inside the pop- up window; Andres, A.N.; Pozuelo, F.B.; Marimon, J. R.; de  Links: to the gift shop or any designated Mesa Gisbert, A. 2012. Generation of virtual site for promotions; models of cultural heritage, Journal of Cultural  3D models manipulation: by Heritage 13: 103–106. introducing a non-commercial software De Luca, L.; Busayarat, C.; Stefani, C.; Veron, P.; environment (like Meshlab) for 3D Florenzano, M. 2011. A semantic-based model-related manipulation; platform for the digital analysis of architectural  Browsing is conducted in a non-plug-in heritage, Computers & Graphics 35: 227–241. environment: Other than Flash, no 3D Duran, Z.; Aydar, U. 2012. Digital modeling of plug-in is required to be installed or run world’s first known length reference unit: The in the background, thus, it is a simple Nippur cubit rod, Journal of Cultural Heritage, way to maintain the system for in press. maintenance and assure the maximum compatibility. Guarnieri, A.; Pirotti, F.; Vettore, A. 2010.  Uses off-shelf software: The Cultural heritage interactive 3D models on the fundamental graphic elements exist up web: An approach using open source and free to administrator management (for software, Journal of Cultural Heritage, Vol. 11, example, Web pages). Web page editing Iss. 3, July–September, pp 350–353. software is used, and no programming is Haval N. 2000. 3-Dimensional Documentation of required. Anyone can learn and Complex Heritage Structures, IEEE MultiMedia, implement the data with ease. Vol. 7 Iss. 2, April, pp 52-56.  Data originality: In addition to original Hernandez, M. 2008. The challenges of preserving point cloud data, it uses the format digital data for World Heritage sites, United down to the basic level, such as the Nations Educational, Scientific and Cultural XYZ file format, which only stores x- Organization (UNESCO), Nice, France October coordinate, y-coordinate, z-coordinate, 29-30. red value, green value, and blue value of each point. Each obj file is prepared in Juming Museum 2011. http:// www.juming.org.tw/. two resolutions for each object: 100000 Manferdini, A.M.; Remondino, F. 2012. A Review polygons and 10000 polygons. The of Reality-Based 3D Model Generation, conservators can convert to different Segmentation and Web-Based Visualization formats or resolutions as needed. The Methods, International Journal of Heritage in the simplicity facilitates migration between Digital Era, Vol. 1, No. 1, pp 103-123. old and new systems, as well as related Remondino, F. 2011. Heritage Recording and 3D conversion and management. Modeling with Photogrammetry and 3D Scanning, Remote Sens. 3:, 1104-1138. 6 Conclusion Sitnik, R.; Krzeslowski, J.; Maczkowski,G. 2012. Digital preservation efforts conducted on stone Processing Paths from Integrating Multimodal sculptures is translated into different forms, which 3D Measurement of Shape, Color and BRDF, serve the purposes of conservation, promotion, and International Journal of Heritage in the Digital public education. The 3D scan-related data and Era, Vol. 1, No. 1, pp 25-44. transformed formats enable a virtual environment with greater interaction and improved photo- Trapp, M.; Semmo, A.; Pokorski, R.; Herrmann, C. realistic experiences for the public and D.; Dollner, J.; Eichhorn, M.; Heinzelmann, M. 99 2012. Colonia 3D Communication of Virtual 3D Journal of Heritage in the Digital Era, Vol. 1, No. Reconstructions in Public Spaces, International 1, pp 45-74.

Figure 1. Stone Conservation Lab, Juming Museum

Granular Disintegration Granular disintegration arises from the rates of expansion and contraction differences of temperature changes, which is in turn due to the minerals on the stone surface having different coefficients of swelling. This phenomenon is often seen in sandstone, the loosened mineral granules of which would usually drop off and accumulate beneath the stone. Block Disintegration Block disintegration occurs when stone repeatedly expands and contracts along existing joints or cracks, during the day and at night. The joints or cracks would widen and multiply, eventually culminating in lithic disintegration.

Exfoliation This occurs when the stone is made up of finer mineral granules, and the rates of thermal expansion and contraction differ between its exterior and interior parts.

Figure 2. Three types of mechanical weathering (Juming Museum, 2011)

Figure 3.3D scanned data with/out image mapping and the photos of the original sculptures

100

Figure 4. Illustrating interactive digital data in a 3D web virtual environment by panorama and models

Figure 5. Illustrating digital data by scaled physical models 3D-printed by a rapid prototyping machine

101

nU-rss i the dark side of the moon

Marc VEYRAT Franck SOUDAN G-SICA G-SICA Art & Communication Research Group Art & Communication Research Group IREGE, 4 Chemin de Bellevue, BP 80439 IREGE, 4 Chemin de Bellevue, BP 80439 74944 Annecy le Vieux CEDEX 74944 Annecy le Vieux CEDEX [email protected] [email protected]

103 Abstract Introduction

There is no other reality than that secreted by the This creation goes throughout two stages. Firstly, se- simulation models. veral signs - we could define as "i+D/signs"(1) - are Jean Baudrillard, De la séduction, extracted and reassembled according different levels Éditions Galilée, collection Folio / Essais, of introspection going from a basic keyword agglo- Paris, 1979, p.23. meration (i-deal) to a personal Facebook’s wall analysis (u-rss+scanner): Where i-deal gives us a quick We are currently developing a project called U-rss snapshot of semantic relationships between queries (http://www.u-rss.eu/) that allows to set up what we and search engines (whether they’re used for texts may call a layer of social portraits, superimposed in or images), u-rss+scanner is a simple gate we have fine to Google Earth. This layer contains several 3D built in order to record a Facebook activity on a ex- modules (the social portraits) which are created with ternal database. Thus, we can get rid of Facebook the help of analysis tools built on top of APIs that point of view and restrictions, write new algorithms Web platforms provide in order to interact with their on large scale of data and extend the Facebook API services. These tools give us the understanding of with more polemical methods(2). During this first someone or the group he relies to, how he is descri- step, we are able to retrieve meaningful descriptors bed, qualified and linked by the Web itself. The pro- that speak for cultures practices, and sensitivities of ject is framed by the idea that the platform speak anyone; a set of signs, allowed by mutual agreement more than the actor himself ; our main purpose being of participants in this project. Then, in a second step, to detach every automated processes that appeared these signs are organized, processed to give rise to glued together on our screen, especially with Face- what we may call modular architextures (informa- book. This joint program, always focused around a tion + signs = in/signs). In other words, we gather specific territory, was applied in Mexico City and signs given by our tools into a 2D graphic we lastly Toluca with a group of students and teachers of the interpolate in 3D. These architextures - real social TEC of Monterey (http://micampus.tol.itesm.mx/) portraits of in/signs that are resulting from cross who have created, during a whole week, a language content, collected information, themselves summa- of signs and shapes understandable by all, regardless rized (compressed?) using i+D/signs - are finally vi- their nationalities or language barriers. sible on Google Earth and instantly legible from RSS feeds.

With the computer, the visible has always a blurred side projected in the shadow, this is the program. Ho- wever, a precision of vocabulary is needed: we tend to approach the relation in a dichotomous way (the light against the darkness, the visible against the invisible, either in the case of the computer: the information vs. programs, codes and algorithms) while we should understand both sides in the gray of the relationship, in the penumbra, in an interface that is capable of retaining traces of the two poles at once. The penumbra is therefore that gray area (the uns- poken subject) where the light illuminating the body (information) is partially intercepted by the program. In this original context, the filtering parameters made from Facebook to build i+D/signs on U-rss will allow us to update and gather particularly rich and Keywords: interesting criteria, which will overlap this field of study by producing new forms built upon, in bet- Google Earth, Facebook, data visualization, data ween existing forms. This work in progress is asso- mining, digital art, cartography, social network, ciated with the fundamental idea that, while seeking API, code, semantic Web. to transform the way we design, publish and share

104 content on the Internet, we inevitably offer new 1- CONVERSATION: the spring principle forms of identity, defined by social logics directly associated with web users and the ability of ma- It is interesting to note that Spiral Jetty imagined by chines to produce quantifiable and interconnected the artist Robert Smithson (1970) "which creates the data. In this idea of "semantic web"(3), it is therefore experience of a constant shift"(6) perfectly sums up here we’re able to review our personality from the this idea of a double point of view emphasized in our intersection of two points of view that seem both proposed U-rss project. With the work of Robert contradictory and complementary: Facebook and Smithson, paradoxically, when a person runs on the Google Earth. pier to the center of the spiral, he moves away from the shore (which might delimit, to him, the boundary While the first proposes a formatted structure where of the land). But the more he approaches the center we have to slip into in order to build a tribe - in (thus theoretically: the idea of having traveled the which we may lose ourselves at any time - the se- farthest on the water), the more the concentric rings cond gives us the illusion of zooming from the col- of the pier separates him from the lake... Similarly, lective to the singular, from global to local levels. in U-rss, if the recovery of a constant mass of data Thus providing a focus on a progressive fold (“re- on Facebook inevitably produces a social portrait pli“ in french) to the peculiar, a deep whirlpool di- from a set of important inter-community connections ving into private space, a bit like a snail retracting (the tribe of the verb be?) - and somehow determines itself to the center of its shell. Three other parameters the avowed desire of a view opened on the outside - in this art and research project still need to be em- this transformation of raw data into corrected data phasized: first, how to redefine the concept of reality seems a priori attached to a result inversely propor- through the device of re-presentation where the tional to the programmed investment: a continuing image (the social portrait on Google Earth) of the interest to assert his limits. image (the set of i+D/signs) of the image (the Face- book tribe) does only illustrate this successive over- The experience is similar to that described above lapping of simulation models? A second line of with Spiral Jetty. What happens when you leave approach will inevitably asked the role of "i-nego- yourself? The hypothesis is to consider the duality tiation"(4) we have to implement in this work since between two points of view apparently contradictory it is a constant confrontation (conversation?) bet- but necessary. This hypothesis can be summarized ween our own desire to interact with a community by one simple concept: the spring principle. On this and algorithms - that already carry ideas and opi- idea, relational design sets to work in U-rss requires nions - behind the interfaces we use to exchange, us to suggest that the development of a dialogue with maybe just like a visual anthropology exercise ap- the Other invariably seeks to build a private space, plied to digital networks. Indeed if an initial scan of only visible within its permeable boundaries... Thus, the Facebook wall is done automatically by a pro- this conversation between the Facebook wall and gram, the raw result is then modulated (corrected?) Google Earth - we could be inclined to initially think by the play of i+D/signs, so the interest of certain as the necessary appropriation of a new living space elements - for us or the owner of the wall - can be (Internet) - would be able to build both the eyes of overstated depending the automatically generated others and the policy of an ego identity. It would gra- links... Finally it is important to notice the rhythm of dually composed the bricks of an autonomous sub- in/signs - leading to the construction (the invasion?) ject, a self sentinel posted secretly under all, who and conservation of the 3D modules on Google would define his elastic limit. And the i+D/signs Earth - which is associated with information gra- would be witnesses of this essential balance between dually - but also temporarily - left on Facebook. tension and elasticity. These limits make - and perhaps anticipate - the indeterminacy of language as The work is constantly evolving, so... a bit like a me- they mark intersections; signal nodes built on net- lody of "Free Jazz"(5) , in which a musician would works that involved a social, outward ego... The so- (re)interpret a known score and adapt it during a cial portrait marks a temporary elasticity limit. temporary immersion in a social body, U-rss develops several recurrent issues raised by the sometimes As a focal point (superimposed to the Google Earth ambiguous relationship relying arts, science and texture), this social portrait is located on the optical technology since the advent of the Internet. axis of a lens: a metaphorical meeting with the Other.

105 But if like him it focuses light rays, it is only possible In U-rss, the transition from the surface of i+D/signs during this process of constant back and forth bet- to the simulation of a 3D volume takes place in three ween the opening and closing of this lens. acts. As we said, everything starts with the information originally reported on Facebook, which are then Jean-Louis Boissier writes about the work "Five Into analyzed and transformed into i+D/signs, and last One"(7) of Matt Mullican: "The virtual here is nei- modeled in 3D... before being placed on Google ther a representation nor a simulation, it is the pro- Earth. In our daily use of these tools, we could think jection of a mental universe topology. In this sense that they represent something important to us, they it has the consistency, the concrete presence of a raw interface things that seem to be linked to reality whe- object, strictly artificial, shaped as a map project, wi- reas they just emphasize the presence of something thout a map of any territory"(8). The correlation ge- absolutely artificial, just because they rely on codes nerally drawn between public and private spaces, which could be re-interpreted, re-framed and re-ex- experienced here through U-rss, has to be interpreted posed. In order to get a wider perspective on our in the sense of a Weltanschauung (worldview). It is daily digital life, we believe that we have to pene- first and foremost a matter of speech: a reversible trate the algorithms beneath the interface so we conversation. This one - theoretically here because could project the proper mental universe (an utopy?) always linked to language - produces the opening to that stays within and that we could only achieve it the Other, organizes, arranges his own mental terri- by melting the territories of what those tools aim to tory of identification by the use of essential filtering map. In other words, U-rss becomes a bit like the devices. These devices cause many levels of inter- Rosetta stone, a collection of words and images pretation needed for this speech to solidify, to sedi- where three languages (of the same text) coexist wi- ment, to close itself gradually until it looks like a thin an i-negotiable phenomenon so we can attach, shack. Because "virtual worlds, as generally devices in some way, the Google Earth territory to Facebook of digital interactivity, remains within the partition, walls with language springs. We are here far from a an art of interpretation"(9). simple "informational acupuncture"(10) set by Mau- rice Benayoun: the tattoo operation designed by successive implants - springs of language - on the 2 - U-rss team, social portrait: Google Earth platform virtualizes it. Musée Gassendi, Digne-les-Bains, ©2011.

106 2- CONSTRUCTION: i-Negotiation sion so. Dantec then describes very well this society of informants, organized as a network of dealers The i-negotiable (tradable information) is a generic whose main product is information itself. Thus there term we use to describe the conversational process, are small dealers, traffickers and barons finally, the the exchanges of contents. This term points the pro- last link in a chain where each branch contains actors cess that engage actors in discursive relations they who try to take profit (trading) of information they maintain towards a resource or any production of the provide the transmission for. It is interesting to note mind, and that needs a construction of its own legi- that the life of all human kind depends on the good- timacy in order to exist, whether it is an intellectual, will of these barons, in the information they allow strategic or simply social one. This fundamental to be traced. Otherwise said, the value of each principle can be stated as: "The information always human life is estimated upon the width of the pipe comes with a price and delivery entities have always of information they are. been able to exercise the right to influence its rate"(11). Information has this inherent quality, an However, if we want to talk about the i-negotiation easily corruptible, playable rate that allows media over the Internet and more especially in the use U- spheres to exist; cost of information seems to have rss proposes to do with Facebook, we have to say a even formatted logical media technology, from their few words about this web social service. The high- hierarchies to their routines. tech platform requires us to include algorithms and programs as influential participants, acting in the ne- The analysis of this business obliges us to describe gotiation process. Decision-making entities that are the whole set of players who can take a part regar- decentralized - fragmented into a network which ding their influences on the information rate. In the overlaps an increasing amount of trading places, so pre Internet era, when dealing with traditional media, that each user can shape an informational sphere of we can consider that the technology actors, the tran- its own, according to his will, where he can access sient channels, exert a relatively little influence all resources the world knows - does not alter the fact considering our problem, the price of information that computer architectures operate with varying de- seems to be mostly modeled, constructed by organi- grees of aggressiveness, influence and change to the zed human agents. In these systems, regulation access to information they solely provide. Quite the means a lot, and the act of piloting information, their contrary, it’s the very existence of these programs switches from one to another is governed by an or- that made possible the emergence of this new para- ganization (an institution in most cases) that is digm the Web media is. If hierarchies are no longer highly hierarchical thought. In this case, if a current visible in such systems, or, to quote Castells, if we (stock) of information sees its stock price raising, it went - now - from a "space of places" to a "space of is relatively easy to trace the chain of transactions flows“(13), it’s not so much because the Internet is that have granted its final value because the players profoundly ubiquitous (trading eras moving at the who cause this are easily identifiable. This principle speed of light until they become a stream), but be- is also sufficient to explain the archaic systems of cause technologies and programs build maps to knowledge transmission, especially when it comes drive the streams, thus forcing us to reconsider both to societies based upon spoken words. the territory and the map logic.

In a science fiction book as Grande Junction by It is necessary to identify these programs, their in- Maurice G. Dantec(12), the archaic regression of influences and their possible corruptibility. Coming formation’s delivery systems is based on deregula- back to Facebook, the geometries of the platform are tion of these latter. We are in 2060; in an information so complex that we are actually unable to emulate society which has seen its distribution organisms our model, without talking about the inside mecha- gradually technicized until every human is connec- nisms of features such as I like it, suggestions, ted through bionic implants to a central machine. groups or fans and what they involved. Information The plot of the book lies in the intrusion of a com- that is negotiated with the algorithmic logic of Fa- puter virus, bugging the central machine and making cebook on one hand, with members of our network every little digital object perfectly mute. This gigan- and the other, and corresponding to the digital iden- tic bug, called the fall in the novel, leads each human tity of the user, presents such permeability towards to a non sustainable use of language, oral transmis- actors (whether they’re technological or not) with

107 which it passes, that its rough edges make it elusive. These scraps of identity can then be classified accor- The malleability of the web forces us then to find ding to their liquid state. Except for what we could any material picture [Gaston Bachelard, 1947] capa- consider as a fossil record – corresponding to the ad- ble to stimulate our thinking. Very influenced by the ministrative profile of a user, an indelible mark thought of Peter Sloterdijk and his trilogy Sphären consisting of a name (an index) and all the profile (14), Bernhard Rieder proposes the metaphor of di- information linked to a tangible existence (address), gital foam to designate the social web and its impact or to any other physical communication tools (tele- on the interactions between web users. Most interes- phone, mail) - if we keep only information on the ting in this article is the concept of membrane both wall, we can measure the liquefaction of informa- filters and interfaces of a bubble containing the ori- tion, its degree of permissiveness. It remains ice if it ginal identity equipment, constantly subject to intru- does not cause reactions, it tends to ice if the reac- sion. "The buble must stay away from others but its tions are removed, it tends toward liquid if it causes extension inland (identity or home of meaning) de- reactions (quantitative approach), it tends to gas if pends on the production of such other space of their the reactions present some form of wealth (qualita- own. Our identity must occur by both relationship tive, social approach: rich network, information is and isolation, or - to speak in Bourdieu's terms - the reviewed by a lot of different friends, themselves game of distinction demands that everyone plays posting personal resources or other hypertext). This it."(15) kind of report to the membrane is selected for U-rss, report that can be envisaged in terms of interface by Under the U-rss project, we retain the concept of extending what this net (work) retains or misses, de- membrane as the (none) place of our experiment, letes or rewrittes. space exchange (of flows) which build relationships between members of a shared network. The importance attached to walls (membranes) of web users is 3- CONSERVATION: i-Documentation not to be looked in the exchanged contents but through the pulse of it, somewhere in between the U-rss provides several computer programs develo- relationship of its owner with its network of friends ped in AS3 and PHP that perform certain tasks with and programmed tools of Facebook. This is not so respect by obligation to the Facebook API. Applica- much upon a qualitative interest that the project was tions are available on the Facebook platform itself born, but from time measurement, from an activity and it is therefore possible to operate them. But they that include rates, pulses and echoing vibrations. We are mostly associated with the development of two consider that a wall is no less a musical score on important protocols: which could be written the digital identity of his - A documentation playing the same role as a cartel owner. Research by Bernhard Rieder and more ge- of a physical artwork, explaining the ontological nerally marine ontology or the aqueous lexicon de- conditions and uses of this program and aesthetics. veloped by Peter Sloterdijk is an interesting basis for - A book of rules defining the limits of acceptable considering the report that information and algo- uses for a particular program and any contributions rithms maintain in such interfaces. On this mem- to the U-rss project in general. brane, we do not dwell on content, or even on hypertextual qualities within them, but on its origin The programs offer shortcuts to perform tasks such first, its liquid state then. as scanning posts of a wall, retrieving images from an album, searching by keywords, etc. The source Thus, activity of the wall can be segmented accor- code for these programs are documented and made ding to the origin of the interface trail it leaves: available for everyone. The progressive development 1 – Algorithm activity: all jobs generated by the sys- of this toolkit may be, as usual with most open tem itself, traces of an activity outside the wall: I and source projects, augmented by contributions from one Other are now friends, I is a fan of ..., I joined other artists, either by modifying the source code of the group ..., I commented link ...). existing applications or by providing new applica- 2 – Author activity: posts on the wall by the owner tions. The purpose of this part of the project is to in- (role of images and hypertext links). volve various artistic actors who do not necessarily 3 – Network activity: post by others, comments, possess the technical knowledge about the use of spontaneous interventions. open source projects. Documentation for a program,

108 although it may be offered in a traditional "Java- as computer art. Thus, two main branches are created doc“(16) format, will have to be associated with a in this documentation: plastic, aesthetic and theoretical discourse, suppor- - The institution branch, designating methods and ting the creation of the latter as it was developed with pointing applications that could be of interest for an an artistic process in mind. institution on Facebook. - The artist branch designating other methods and Documentation for these programs is then proposed applications of interest in an artist's personal ap- as an alternative reference to the official API proach. (http://en.wikipedia.org/wiki/Application_program- In addition, to allow insertion of projects brought by ming_interface) of Facebook. It means that there will other contributors, creators of the project will offer be no question of relationship between functionality a book of rules that one must comply. This book de- and algorithmic vocabulary(17) anymore, but bet- pends on the game levels to which the contributor ween possibility and artistic problematic. From the wishes to submit (transparency of inclusion) and a Facebook’s API, some terms will be extracted, refer- classification of the action/shape dialectic (reminis- ring to very specific methods to achieve what may cent of the universal form): action on Facebook and be called a paradigm of the existing digital, tracks shape in Google Earth. Two rules are however ne- that can combine several possible methods but must cessary to thin the transition between the selected ac- make sense as artistic approach inside U-rss. We are tions reserved for the development of the social then facing a distinct material of the Facebook API, portrait and its translation into a visible shape an interactive language-specific project. The goal is (in/signs) on Google Earth. not to develop an efficient and functional API, but to offer entry points for uses or developments of ap- 3 - U-rss team, social portrait: plications that need a clear data model to truly work UƒHo! (Franck-)i goes to all i WWW, ©2012.

109 Transparency mations using semantic technology that will translate The concept of transparency is selected here to ex- word into magnitude of the in/sign, altitude or opa- plain the report that levels of implications (gray, city, its value against the U-rss 3D module, its value pink, red) will maintain with the interface. In a trans- short... The Fine Arts classification of Etienne Sou- parent relationship with the production of social por- riau [Souriau 1920], colors / stamps / emotions chart trait, there are no other interfaces than the Facebook of Kandinsky can then present a great utility. The platform. An application is developed through the whole being is to identify the relationship we have contributor's (the artist or the institution) wishes; he with our digital twin and network. Some play on the then installs it on its own wall. As mentioned before, concepts of private / public space, thus moving a cur- the scanner merely exists to extract information that sor throughout a veiled / hidden rule, others measure the contributor deems necessary, depending on his the value of information (useful / useless), so the goal and how he thinks himself on networks. But it more an information they share raises reviews, the remains invisible to his network, so no “friends” are more their ego inflate, the more the spiral that draws aware that their interactions with that wall are used a module extends. It may also play on a different elsewhere. It is the gray level, the spy in the belly of axial relation. The inclusion of a third dimension Neo who tracks, records, agglomerates and translates provided by the use of Google Earth also allows a the information to us. The pink level corresponds to glimpse of a different relationship in terms of inclu- the setup of an application by other users, which pro- sion. The in/signs may develop on a horizontal plane vides additional information, more sensitive, some- (x / y) so they create a kind of forest, or on the z axis times more accurate; the spy eventually ends up to according to the depth with which the contributor present some kind of ubiquity. The interface remains appears on Facebook. unique, it is still represented by the Facebook platform and the essential operation of translating the data into shapes (in/signs) can work without the use Conclusion of an interactive interface. It is the contributor who programs his algorithms, thus developing a hidden If, as Jean Baudrillard pointed out in the 70s, mea- relationship in his action/shape dialectic. So far, ning takes precedence over the signified, then the there are no other interactions on the social portrait verbal or visual symbols that represents things and that those provided or authorized by Facebook. The real objects are detached from their referents: they red level involves the outsourcing of selected re- are referents (because they are digital). Not only the sources on Facebook in an intermediary layer such images replace the real but also they define this real. as a database. The artist then proposes an interactive Thus, we do consume neither objects nor situations, artwork so that the raw data extracted from the API but the meaning of these objects reported in situa- are replayed, open to corruption. The red level dee- tion: we consume information. In any presentation pens the relationship with the digital identity and space (or exposure space), if the context changes the would, through the creation of an intermediate inter- perception of the image, it is especially the re-cog- face, give a hand on data so that an entire private nition of it that has already changed the perception segment slips into public space. This level designs a of this image stored in its context. With information, major “advocacy” of U-rss : the right for anyone of the boundaries of territories settled by successive us to retrieved, copy, share any data we added to Fa- codes or signs articulate our progress. We are both cebook outside of it.(18) eyes on the dish of real.

Shape As for the moon, what we see is only what is looking In the same way as we faced given solid and / or li- at us. The reality becomes reported reality in an area quid data, we are facing fossil or sandy in/signs. that is perceived as an area only because it is appoin- There are specific actions on Facebook, for example ted. And this space should allow us to collect all the I like it could be considered as fossils, binary layers of construction, i material distance (the dis- in/signs, episodic wave of lean interest, planted tance made by the information material) between around a social portrait and or we may face sandy material signifiers, structures and meanings of these in/signs evolving such as reviews or measurable in- joints. But with U-rss in the dark side of the moon formation we can extract, hypertext linking to a web other issues also seem to point through the intersec- page opening the post. We could scale these infor- tion of these two platforms: Facebook and Google

110 Earth. The coefficient of proposed reality suggests ration around a shared real. In summary, if U-rss in immediately a doubt about the visually observable, the dark side of the moon is mostly based upon the what our cognition understands through technology. share of a real: U is M-i @lf-real friend, with the source of this event, our voluntary presence on the With Facebook & Google Earth - even if the latter landscape of the Other is only a crystallization of has implications with observable reality: for example elements that become meaningless when shared by the real presence of the ©box and the IAE Savoie a form of universal language, somehow a joint pro- Mont-Blanc leads to a kind of confrontation between gram... two nested observation, a process that can be called a “réellité” (actually a temporary and partial reality).

Any integration of digital 3D elements (ingredients?) Marc Veyrat / Franck Soudan, 2012. on a programmed system which thinks itself as a projected photography (Google Earth) also disrupts the readability of the system. Finally, does this project can then be regarded as a prism, a (no) place, or simply the place of language? To see through; all the richness of this (false?) crystal, while relying only upon the choice of ingredients, work plan and statement of the recipe... In U-rss, if the physical location to the project is fundamental, it gradually dissolves in its digital identity. We do not choose a place for what it says about the reality it represent, which means: city names, roads, neighborhoods, but because we need a visual boundary, a proper portion of the globe where our investigations are echoing the map. The same logic is applied to existing buildings that needs to be created and implemented on Google References Earth because they become a part of U-rss. For example, during the invasion of a geographical space Norbert Wiener, Cybernetique et société, Éditions - in this case Annecy-le-Vieux - with any module des Deux-Rives, Paris, 1952. (the above IAE Savoie Mont-Blanc that is already Pierre Lévy, Qu’est-ce que le virtuel ? Éditions La invaded by players), we do not include the architec- Découverte & Syros, Paris, 1998. ture of the real building with care in order to simulate Michael Rush, Les nouveaux médias dans l’art, Lon- his presence on Google Earth (the 3D model is blue, don, Éditions Thames & Hudson, London, 2000. transparent and has no textures), but to identify the Claude Rutault & Frédéric Bouglé, La fin de l’objet institution we, as searchers, depend. So any indivi- fini (entretiens), Éditions Joca Seria, Nantes, 1992. dual who is living around or who discovered the pro- Paul Virilio, La machine de vision, Éditions Galilée, ject is able to produce (digital) i-real. This digital Paris, 1998. report seems to present two interesting points: the Samuel Beckett, L’image, Les Éditions de Minuit, content is intended to position the institution or in- Paris, 1988. dividual with respect to the norm. This means that a Bruno Latour, Why Has Critique Run out of Steam? difference can be measured depending the hosts of From Matters of Fact to Matters of Concern, Critical the place and if all these productions are linked to Inquiry Vol 30 n° 2, Retrieved January 31, 2011, the existence of a physical territory (a building, a from http://www.bruno-latour.fr/articles/article/89- place...), these productions of information (i.e. com- CRITICAL%20ENQUIRY.pdf, 2004. munication) probably correspond to operations of Marc Veyrat, ©box - interMade : l’interstice comme deterritorialization (Deleuze). In other words, this facteur de lien social, in Rencontres entre artistes et digital format forces us to reconsider the existence ingénieurs autour du numérique (mobilité et glocal- of the territory as an open source program. What ité), collective work under the direction of Yannick does that mean? Perhaps that an idea, a thought or Fronda & Grégoire Courtois, Éditions L’Harmattan, an event that produces any i-real is never a collabo- Paris, 2011.

111 ( 1) These common signs, which we call i+D/signs are not look a little ridiculous? Your life in this format?” Zadie simple translations of words. These are cuts of informa- Smith (2010), Generation Why? Retrieved February 4, tion using recurrent signs (e.g. the i of information that 2011 from the web : http://www.nybooks.com/articles/ar- any person intends as a signal), resulting from clues left chives/2010/nov/25/generation-why/ by the use of certain words, their circumstances, their positions in a message on the wall of Facebook (the D that (3) "How such systems will they live, and when they will stands for “definition”)... These i+D/signs are used to in- begin providing meaningful answers, begins to be the tentionally cause interactions between words, colors and topic of many researchers and experts." says John Mar- objects. "In a world actually overthrown, the true is a mo- koff for the New York Times. For Markoff, this Web 3.0 ment of the false" says Guy Debord in his famous best sel- is based on the excavation of human knowledge, as Goo- ler La Société du Spectacle (Éditions Gallimard, Paris, gle has operated within its Page Rank (who plays the links 1992, p.19). However, as pointed out by Jean Baudrillard of a web page to another as a vote). And to give a sum of to the question of the sign / simulacra, now, in this really examples to his thesis: "We're going to a Web of connected reversed world, it is the false that has become a moment documents to a Web of connected data" said Nova Spi- of the true. Thus, on our computer, we're looking through vack, from Radar Networks, a start-up company that ope- the window / screen that is not really a window on the rates the content of social network websites, and who world, something that exists, which is not true, but which reported recently on his blog, how fed up of web 2.0 he refers to something real or at least to what we know about was ("Destroy the myth of Web 2.0"). KnowItAll, from a it, our certitudes. For example, in the i material society, research group at the University of Washington, extracts the image of green grass will be chosen because we all and aggregates information from products testing web- believe, we all know intuitively that the grass is green. sites to provide understandable information to the user. Consciously, the false then gives the illusion of the true So today, for information about travel, you need to review and when the image becomes true, then the société i ma- long lists of comments gleaned from the web. With web tériel really starts... Jean Baudrillard writes in his book 3.0, the system will classify all the comments and will De la Seduction (Editions Galilee, Paris, 1979, p.78): "any find, by cognitive deduction, the right hotel for your par- talk of sense wants to end up appearances, that is its lure ticular need. "In its current state, the web is often descri- and deception. But also an impossible undertaking: the bed as being in the Lego phase, with many different parts speech is inexorably linked to his own appearance, and able to connect to each other. Those who bear the vision thus the stakes of seduction, and thus its own failure as a of a next phase, web 3.0, see it as an era when machines discourse". The speech would be a network of identifiable will start to do seemingly intelligent things. […] It is clear signs potentially related to ideas - what we call i+D/signs, that human knowledge is more exposed to machines than whose meaning is still subject to the shape, appearance it has ever been." says Danny Hillis of Metaweb. From and charm of these signs... artificial intelligence systems, such as Metaweb or Cyc, which combine basic and conventional rules for of web (2) On social networks, we always face multiple active content analysis, could afford to operate always better this speeches all meted together on a single interface. Our incredible database that is the current web, to provide ans- message is written to be shared on a network of contacts wers to complete questions. Unless, thinks the Research but when it becomes visible to them, our content is aug- Manager of Yahoo!, salvation come from the aggregated mented by the code and the discourse of the development user intervention: "With Flickr you can find images that team behind the platform. This inclusion produces more a computer could not find. Problems that we are challen- damages to our words than a simple and necessary inter- ged for 50 years, suddenly became trivial." Guillaud Hu- operability process: it changes the design of meaning. Fa- bert (2006). Vers le Web 3.0. Retrieved January 30, 2011, cebook does not only provides a tool with which we share from http://www.internetactu.net/2006/11/21/vers-le-web- our identity, it tells us how we should interact on the Web. 30/, (translation: Franck Soudan). That is precisely what Zadie Smith points out: the fact that Facebook is merely a Mark Zuckerberg production. (4) Franck Soudan, Collectifs intelligents et algorithmes, “Shouldn’t we struggle against Facebook? Everything in l'i-négociation sur Internet, in H2PTM'09, actes du col- it is reduced to the size of its founder. Blue, because it loque, Editions Hermès, Paris, 2009. turns out Zuckerberg is red-green color-blind. “Blue is the richest color for me - I can see all of blue.” Poking, be- (5) Free jazz normally retains a basic rhythm, but without cause that’s what shy boys do to girls they are scared to regular meter, with sudden accelerations and declines, as talk to. Preoccupied with personal trivia, because Mark the sea swell. Often the musicians of the same orchestra Zuckerberg thinks the exchange of personal trivia is what are playing on different tempos. Wikipédia, Retrieved Au- “friendship” is. A Mark Zuckerberg Production indeed! gust 8, 2009, from http://fr.wikipedia.org/wiki/Free_jazz. We were going to live online. It was going to be extraor- We can read about this relation between Jazz and U-rss : dinary. Yet what kind of living is this? Step back from Marc Veyrat & Franck Soudan, Le Jazz fabrique t’il en- your Facebook Wall for a moment: Doesn’t it, suddenly, core des images ? in Les Territoires du Jazz, collective

112 work under the direction of Jean-Claude Taddei, Éditions (13) Manuel Castells, The Information Age : Economy, Presses de l’Université d’Angers, 2011. Society, and Culture: Volume I, The Rise of the Network Society, United States: Blackwell, 1996. (6) Centre Pompidou : Traces du Sacré (2008). Retrieved January 30, 2011, from http://traces-du-sacre.centrepom- (14) Peter Sloterdijk, Sphären, Germany, Francfort: Suhr- pidou.fr/exposition/oeuvres_exposees.php?id=31 kamp Verlag KG, 2004.

(7) Matt Mullican (1991), Five into one, in Artifices 2. (15) Bernhard Rieder, Membranes numériques : des ré- Retrieved January 30, 2011, from http://www.ciren.org/ar- seaux aux écumes. Paper presented at the PHITECO se- tifice/artifices_2/mullican.html minary, UTC de Compiègne, 2008.

(8) / (9) Jean-Louis Boissier, La relation comme forme, (16) Javadoc is a tool for generating API documentation l'interactivité en art, Switzerland, Geneva: MAMCO, in HTML format from doc comments in source code. 2004, p.64. (17) For example, one of the most widely used methods (10) Maurice Benayoun, Colloque H2PTM'09, Session of the Facebook API method is the one called Users.ge- Art des Nouveaux Médias, table ronde, laboratoire CiTu, tInfo. It can recover some information about the user. But Université Paris 8, Paris, 2009. does this raw method make sense for the U-rss project?

(11) op. 3. (18) This right is actually not provided by the Facebook Platform Policies (http://developers.facebook.com/po- (12) Maurice G. Dantec, Grande Jonction, Albin Michel, licy/) thus making our scanner application legible for shut Paris, 2006. down.

113

Connected Creation: The Art of Sharing

Florent DI BARTOLO EA 4010, Arts of Images and Contemporary Art, University of Paris 8 & G-SICA, IREGE, University of Savoy 4 chemin de Bellevue – BP 80439 74944 Annecy-le-Vieux CEDEX France [email protected]

Abstract used in the late Middle Ages to confer order to memory. Coined as “art of memory”, this system Launched in 2006 by the CiTu-Paragraphe under was used from its introduction by Simonides de the artistic direction of Maurice Benayoun, the Ceos around 477 BC up until the 17th century to In/Out network (aka The Art Collider) is an open structure memory impressions, improve recall, platform that uses peer-to-peer technology to and contribute to the combination of ideas (Yates, enable real-time data exchange between art 1975). However, we might ask what type of installations. It has been featured in several system do artists use to structure and arrange exhibitions over the recent years. This paper elements today? proposes to analyze its artistic potential: First, by They use database systems within which they describing and characterizing the type of configure spaces and establish connections connections that the network allows to establish between different data sets. Online databases between peers. Second, by analyzing how data is make it possible to retrieve data using advanced shared and transformed by artists according to a search criteria, in order to use their content to poetics of data. Finally, also by analyzing how the create dynamic compositions. Gregory Chatonsky, network presents itself and how the exchange of for example, has created over the last ten years data has been made perceptible to visitors both several web applications such as The revolution locally and globally through art installations and a took place in New York (2002) that use Flickr to monitoring system. dynamically retrieve vernacular pictures - taken by multiple individuals all over the world - in Keywords order to associate them with other images and textual statements (in a slideshow format). Peer-to-peer network, shared creation, poetics of Database systems are also used to configure data. virtual worlds such as the one in which the art installation called The tunnel under the Atlantic 1. Introduction (Maurice Benayoun, 1995) allows to dig. A database system is used to manage the visibility of Art does not evolve in a separate sphere, a collection of pictures in a virtual world based on preserved from reality. Art is transitive and the time that each user spends to contemplate interacts with social space. It establishes relations, them: The more time a user spends in front of a and configures spaces. In this regard, over time, picture, the more chances he/she will have of artists have used several techniques such as finding pictures belonging to a same category. perspective to structure spaces of representation. However, database systems do not only make it For example, during the Renaissance, painters possible to build dynamic compositions and used perspective to organize “the prose of the virtual worlds. They also help to organize social world” (Foucault, 1966) by placing human figures networks: both centralized and decentralized. inside a unified space. At its origins (1430-1460), In centralized networks, databases tend to be perspective had a political dimension and was used to collect large amounts of data and to secure used as a tool to build the scene of history and their access. Thereby, they allow organizations to civic freedom (Arasse, 2006, p. 133). Perspective control and monetize access to their content. Yet, represents a system that differs significantly from databases can also serve to collect metadata in the system of places and images that was widely order to facilitate the exchange of data hosted by

115 peers. Databases are often used in this manner to streams. The terms of the exchanges are based on collect metadata associated with files that are the individual artists’ personal interests. Each available in peer-to-peer networks: in terms of installation effectively reflects a different artistic connection and communication between peers, position and proposes a unique experience of the peer-to-peer networks offer different possibilities network thanks to different types of data streams. from centralized networks. In a peer-to-peer In this regard, the In/Out network allows not only network, users interact directly with each other to to share video streams, but also to share audio and exchange streams of data without having to first raw data streams. In addition, artists can use these obtain permission (from a central entity). Many streams of data in many ways. researchers working in different fields have used In order to establish meaningful connections this technology over the last thirty years to between the different installations, over the last communicate and collect data. For example, in years the CiTu has organized several meetings 1997, the Space Sciences Laboratory at the between artists and programmers. For example, University of California used peer-to-peer ahead of the second exhibition of the network that technology to launch a distributed computing took place in 2008, the CiTu organized a one- system aimed at analyzing radio signals, searching week workshop to discuss the projects and their for signs of extra terrestrial intelligence with the associated streams. The workshop enabled each help of a large Internet-based community. This artist to adapt their project to the needs of the project, called SETI@home, “captured, as stated, others participants. The workshop also made clear the public’s imagination and achieved tremendous that the network needed to be organized. It was popularity” (SETI@home Classic: In Memoriam, therefore decided to use a server to keep track of 2005). However, artists have also used peer-to- each stream generated in order to make these peer systems to collect and exchange data. They accessible to everyone as soon as they would be may even have used this technology to create available. original networks: networks of “shared creation” Being a peer-to-peer network, the In/Out that allow new types of interactions between network allows direct connection between peers, themselves and their audience. but in order to provide access to all the streams Launched in 2006 by the CiTu-Paragraphe, the generated, the network also keeps track of all the In/Out project is one of these attempts to create a individual streams. Each data stream of data is network dedicated to artistic creation using peer- indeed indexed in an online database. When to-peer technology and a database. The In/Out queried, the database releases information about network aims indeed to create an ecosystem of all data streams currently available. None of these artworks by making the exchange of data between are actually stored. They are only indexed by the artists possible in real-time. The artistic potential database that collects and shares metadata such as of this type of network is addressed hereafter. their name and their URL, that is to say metadata First, the types of connections that the network that can be used to identify them. However, the allows to establish between artists are defined. database is also used to store metadata other than Second, several artworks that have been designed name and URL so as to make it possible for users especially for the In/Out network - today called to search for streams matching specific criteria. Art Collider - are described. Finally, the relations Thus, for each stream, the database records a that exist between a connected artwork and the description, a screen capture, and a list of tags. network itself - considered as whole entity - are The database also provides the name of the person examined in order to better understand how they at the origin of each stream, allowing, for work together. example, to retrieve dynamically only the data streams produced by a specific person or tagged 2. Establishing a connection as “bright”. It is not necessary to query the database in order The In/Out network is the result of a collaborative to establish a connection with another artist. work between different artists and programmers. Anyone can directly ask for the data stream Several teams were set up which were responsible produced by another art installation provided they for creating art installations that shared data know its URL. However, the database plays a key streams in real-time using a same system of role by enabling artists to search in real-time for communication. However, the different streams of data that match advanced search installations do not share the same type of criteria, so that they do not have to rely on chance

116 to obtain a stream that corresponds to their needs. installation (currently connected to the In/Out In this respect, the database offers the possibility network). Indeed, the Driving installation uses an to create connections that are both dynamic and external audio stream to control the speed of the meaningful. Moreover, the qualities of the car displayed on the screen: This higher the sound connections that can be established also depend level, the faster the car moves and conversely the on the number of streams available at the time that lower the sound level, the slower the speed of the a request is initiated. Indeed, the network needs to car will be. The car may even come to a stop if the reach a critical mass, that is to say a sufficient stream connection is lost and no other audio number of peers who generate their own stream of stream is available. The installation does not show data. To reach this stage, the CiTu has launched a a movie with a beginning and an end. Instead, it wiki platform that provides several software plays the same dozen of video sequences in a programs such as the Floximer: The Flomixer loop, showing a car travelling on an endless road. enables anyone to join the network by providing a The audio stream retrieved by the installation is video stream that can be the result of a also used to produce car engine sounds. Created compositing process (Getting Started, 2011). mainly by altering the pitch of the audio stream, The In/Out network uses the database in a the sound of the car engine confers to the Dodge manner similar to the way that Internet services Challenger a real presence inside the projection such as Napster have utilized their own database. room. Developed by Shawn Fanning in 1999 to offer the The Driving installation only shows outside possibility for all to share their MP3 collections views of the pony car. Visitors can not see the online, the Napster platform also used a database inside: they are invited to sit on a bench in front of to keep track of each file available on the hard the screen to watch the car drive by closely at drives of its users in order to facilitate their different speeds. Moreover, in order to establish a access. By making copyrighted materials available more direct contact between the visitor and the for free, Napster defied - until its closure in 2001 - network, the audio stream retrieved by the the rules of the market economy. In the same installation is also used to flash the car lights. By time, the Napster platform contributed to the doing so, the installation aims to reveal the creation of online communities based on peers’ presence of the network behind the image, interests, allowing a new relational sensibility to enabling visitors to imagine all the streams of data develop among young people. The In/Out network that are being generated and retrieved shares these aspects with the Napster platform. simultaneously by the different installations that However, unlike Napster, the In/Out network is are part of the network. not used to share copyrighted materials. It serves to coordinate the circulation of data streams In addition, in order to retrieve an audio stream, generated in-real time by artistic installations. the installation generates its own stream of data. The installation releases the video frames that are 3. Connected artworks being projected inside the projection room in real- time, thereby offering others artists the possibility To further examine potential relations between to use them as raw material. As an example, the different art installations belonging to the In/Out video frames made available by Driving could be network during an exhibition, two specific used by the computer program Crystallizer. Born examples are presented. The first one is an from the collaboration between Robin Gareus and installation called Driving that we created Joseph Nechvatal in 2009, Crystallizer is a ourselves in 2008. Driving is a video installation program that attacks video frames by using that shows a pony car driving through the night. colonies of viruses. The program is inspired by The car (a 1970 Dodge Challenger R/T) was the work of mathematician John Horton Conway modellized and rendered using 3D software. The on artificial life (particularly Conway's Game of frames displayed by the installation are Life). It uses video frames generated by completely computer generated and are projected installations within the network as a host onto one of the four walls of a small and dark environment for viruses that feed on their colors, projection room. Yet, unlike frames projected in a and multiply until they have completely altered classical movie theatre, the frames projected by their code and original appearance. Crystallizer is Driving respond to an external signal. This signal one of the programs that Joseph Nechvatal has is a data stream generated by another art used throughout his entire career to influence the

117 outcome of an image thanks to autonomous agents another. Yet, inside each exhibition space, how (Sikora & Nechvatal, 2011). Yet, unlike other are the connections that are being made between computer virus programs that Joseph Nechvatal the different installations that belong to the has used previously in his viral works, the network made visible? Crystallizer infects an entire network. The attacks In order to make the network visible to each are not limited to a collection of pictures visitor, the CiTu has developed a monitoring previously selected by the artist. They system. This system uses the information contaminate an entire living network by producing collected by the database described previously, to and proposing to the other peers, transformed and show in real-time all the connections that are degraded streams of their own data. being established or closed between peers. The The In/Out network is characterized by a monitoring system has a structuring effect on how continuous displacement of data from one peer to visitors understand the structure and dynamics of another, allowing artworks to invest a space the network. It shows what kinds of streams of without barriers. Thanks to the network artworks data are being shared and where each installation are not confined to an enclosed space, they can be is located, enabling viewers to choose between the distributed. This changes most of the rules that different exhibition spaces. Presented in all apply when organizing an exhibition. As Geert exhibition spaces, the monitoring system proposes Lovink has stated, new media are increasingly several views of the network called distributed and require a new aesthetics that “Connections”, “Physical and Metaphysical moves beyond the concepts of form and medium. Locations”, “Sociability and Nervousness”, They require namely “a distributed aesthetics” “Treeview and Influences”, “Affinity Groups” and that takes into account simultaneously “the “Flux and Reflux”. Each view offers a different dispersed and the situated, the asynchronous representation of the network making it possible production and multi-user access to artefacts on for the visitors to observe many of its the one hand, and the highly individuated and characteristics. The “Physical and Metaphysical dispensed allotment of information, on the other” Locations” view for example places all the (Munster & Lovink, 2005). selected installations inside circles that represent all the exhibition spaces. This view thus allows 4. Distributed creation visitors to observe the data exchanges that are being made between the different exhibition In recent years, the In/Out network has been spaces. On the contrary, the “Connections” view presented on multiples occasions. In 2008 and does not take into account the location of the 2010 at La Bellevilloise in Paris for example. Yet, installations. By representing all the streams of the exhibition of the network does not generally data that are being exchanged between the occur at one place. The exhibition typically takes different art installations it draws the viewers’ place simultaneously at several locations in order attention to the connections established directly to reflect the distributed architecture of the between artists. The view even specifies the type network. For instance, the artworks selected for of streams of data that are being transmitted (raw the In/Out x.0 exhibition (that took place in 2008) data, audio or video) by using different colors to were distributed between several exhibition display them: Each stream is represented by a venues located in Paris (like La Bellevilloise) but colored line that connects two installations also in Montreuil at the premises of PétaHertz (themselves represented as cells). studio and at the Maison Populaire. An interactive The monitoring system helps to map the network installation connected to the network and created by drawing a landscape of connections. However, by Yves-Marie L’Hour and Benoit Meudic was the map is not the network. A network is indeed even exhibited at the Cube (Issy-lès-Moulineaux). opaque, ubiquitous, and non-formal. It cannot be The distance that separates each exhibition space reduced to its representation. It must be from the others, confers a physical dimension to apprehended within the complex ecology in which the network. Each exhibition space helps to define it is developing, that is to say not merely with the the presence of the network in the city by monitoring system that only makes certain types covering different territories. The exhibition of connections visible. The smooth and seamless spaces represent points of connection. They lines that the monitoring system displays cannot enable visitors to enter the network, to follow the account for the human efforts required to create path taken by data to travel from one place to links between each physical installation or to

118 maintain them. The views of the monitoring has to be entered and experienced through system work only in relation to the installations multiple views, as a visitor or as a participant. that are being presented in each exhibition space: The monitoring system is in charge of the global 6. Acknowledgements representation of the network, whereas the individual installations allow to interact directly We wish to extend our thanks to all the members with them, to observe how data is being moved of CiTu-Paragraphe who have participated in the and transformed. The poetics of data that the activities of the In/Out network. We are especially network carries is not fully displayed by the grateful for the support of Julien de la Fuente and monitoring system. It has to be experienced by Robin Gareus successively in charge of the entering the network, that is to say by interacting network development under the artistic direction with the exhibited installations, and visiting of Maurice Benayoun. Our thanks go also to all several exhibition spaces. Unlike the monitoring the artists who have collaborated with us. system, the installations do not offer access to a global representation of the network. They allow References to capture what is not yet described, what is not yet visualized, as illustrated with the flashing Arasse, D. (2006). La règle du jeu. In Histoires de lights in the Driving installation. The installations peinture (pp. 125-138). Paris: Gallimard. also enable viewers to observe how the data Foucault, M. (1966). Les mots et les choses : Une streams are progressively transformed as they archéologie des sciences humaines. Paris: move from one installation to the next, in an Gallimard. endless relay. Getting Started. (2011, 12 22). Retrieved 06 08, 2012, from The Art Collider: 5. Conclusion http://wiki.theartcollider.org/get_started Munster, A., & Lovink, G. (2005). Theses on The In/Out network is not only a tool. At a time Distributed Aesthetics. Or, What a Network is where artistic presence over the Internet tends to Not. The Fibreculture Journal (07). be standardized, it is a collective experiment that Sikora, S., & Nechvatal, S. (2011). Le projet proposes an original way of being present as a Computer Virus. In Z. Kapoula, & L.-J. Lestocart, connected artist on Internet but also in an Esthétique et complexité : Création, exhibition space. The In/Out network makes it expérimentations et neurosciences (pp. 67-84). possible to imagine new types of collaborations Paris: CNRS. and external participations in favor of a poetics of University of California. (2005, 12 15). collaboration. The In/Out network is an open SETI@home Classic: In Memoriam. Retrieved 06 space with no barriers that invites one to be part of 05, 2012, from SETI@home: its making, part of its development and change. http://setiathome.berkeley.edu/classic.php The network does more than simply manage and Yates, F. (1975). L'Art de la mémoire. (D. Arasse, map data. Potentially it enables anyone to play a Trans.) Paris: Gallimard. role in its organization (structure) by proposing a stream of data or by developing a computer program that uses and transforms data streams shared by other peers. However, like any network, In/Out does not constantly grow. The life of the In/Out network is organized around exhibition projects that bring together groups of collaborating artists. The In/out network is a continuous emergent project characterized by a distributed aesthetics that invites one to embrace the surrounding flow of data. For all these reasons, the In/Out network can be considered as an artistic proposition in itself, a collective artwork. Indeed, the In/Out network does not only connect artworks, it also embodies an artwork that

119

Cinema Beings

Alain LIORET Arts et Technologies de l'Image Université Paris VIII France [email protected]

Abstract [4] Gilles Deleuze [5], Sergei Eisenstein and Jean Epstein. The Cinema Beings are a complex ecosystem of Indeed, in addition to producing films and artificial beings whose goal is to create artificial experimental art, the system is intended to films. This system is on one hand primitive and demonstrate some practice of film theory. It also chaotic, and on the other hand evolved and self- opens new avenues of research into the use of organized. Not only does it produce films aesthetic rules, including the widely used of divine independently, it also implements theories of proportion. Quote Bergson [4]: "The only aesthetics and the rules of cinema… cinematic consciousness, not the viewer, or the Cinema Beings are based on a huge Genetic hero is the camera, sometimes human, sometimes Algorithm, and use some Cellular Automatons for inhuman or superhuman”. producing rules of Live, and rules of Art pieces.

2 Relative Works Keywords Cinema Beings produces some artificial cinema, Artificial Life Art, Cinema, Generative Art. and we can go back to the Italian Futurist movement as the first historical attempt to see a 1 Introduction calculated movie with the film "Vita Futurista" Artificial Life is a science studied by computer directed in 1916 by Ginna and Corra. [8]. scientists, and sometimes picked up by artists to Subsequently, many artists have made works of create new art forms. However, most of the time, experimental and calculated films, including Oskar artistic works are based on visualizations of Fischinger, László Moholy-Nagy, G.W. Bitzer, artificial worlds, as in the work of Karl Sims [1], Alexander Alexeieff with Claire Parker, and Laurent Mignonneau and Christa Sommerer [2], Norman McLaren who made the first experiments Jon McCormack [3], etc. The Cinema Beings in new cinema (with animation). We must also world is different in that it is not even observed, mention Ben F. Laposky, Mary Ellen Bute, Karl nor regarded as a work of art, but instead it Otto Götz, and the first artists to use computers to functions to produce feature films. Cinema create films with machines (including Charles Artificial Beings are artists, who by working Csuri, James and John Witney, Stan Vanderbeek, together, following aesthetic rules and major Larry Cuba, Ken Jacobs, Jean Painlevé, etc.) [9]. theories of art history, and thus allow new types of films to be created. Cinema Beings are the elements of a complex System, a huge ecosystem inhabited by artificial beings, making pieces of cinema together. The original idea for the creation of Cinema Beings from a study of great philosophers and film theorists highlighted the organic composition of cinema and its ability to be a machine. The foundations of the system presented here are therefore based upon the texts of Henri Bergson

Visual Music by Oskar Fischinger.

121 3 Algorithmic Cinema 4 Art and Artificial Life Barbara Lattanzi [10] has written excellent texts Artificial Life has already been used by many on film and has created computational and artists in different fields. Mitchell Whitelaw [14] experimental tools, but to better understand has produced an excellent overview and a Cinema Beings’ achievements, we must look at the complete history. In particular, many artists have work of Jim Andrews in regard to dbCinema [11]. used art to create evolutionary new works, dbCinema is an online graphic synthesizer that can following the work of Karl Sims, with William produce real-time painterly cinema. It is a very Latham, Steven Rooke, Jeffrey Ventrella being good work, with experience based upon Kandinsky among the most well-known. paintings, and many others. We can compare his Many worlds populated by artificial beings have work with some experiments made by Lev been proposed, with a common point being the Manovich (especially The Soft Cinema) [12]. worlds that we see and which are in fact, All of these works are based on algorithmic considered as pieces of art. This is the case for cinema techniques. My system for Cinema Beings example, with the ecosystem EIDEA by Lovell is also algorithmic, but is more related to artificial and Mitchell, and Iconica by Troy Innocent. life and goes further in a generative process. However, the work of Jon McCormack [3] and Other interesting work is that of Iain Lobb [13] more specifically, Turbulence and Eden are the which attempts to explain how generative film closest Cinema Beings System ever created. might form a cinematic language. His very interesting approach is about “microworld cinema”, which focuses on the algorithmic 5 Artificial Beings Designers simulation of characters. The uniqueness of our system is that it is an We can also mention the excellent work done at ecosystem populated by artificial beings whose MIT Media Fabrics on Interactive Cinema by world is not seen as a result of the work, but whose Glorianna Davenport and her research team [27], collaborative works produce arts (images, sounds, and the work of Gordon Pask on interactions of movies). These creations are directly connected to actors [29]. Finally, we have to compare our work our real world. to the research made by Stefano Bocconi, Lynda We pop up on "the other side of the mirror" and Hardman and Franck Nack on generating video the final result that we see is an artistic documentaries [28]. Their work is based on interpretation of our world, our ideas, and our semantic graphs for making relations between knowledge through artificial beings, albeit videos. Our work is a system based on relations primitive, but capable of self-organization, and between media elements, but with some evaluation especially sensitive to rules learned methodically. based on a fitness value in a genetic algorithm All the Beings are builded from the same class process. object, with the same DNA at the beginning.

5.1 Plant Beings Our first work in this direction was the creation of Plant Beings [15]. They have methods which are based on L-systems and generations from cellular automaton. These artificial plants are genetically advanced, but at this level, we were still facing an artificial world that is only watched and not creative in itself.

5.2 Light Beings Plant Beings are not integrated into the Cinema Soft Cinema by Lev Manovich Beings system (because of their lack of creative vocation). This is not the case for their direct successors: Light Beings. These are built on the same principle as the Plant Beings, but with only one goal: producing light and shadows in the

122 world in which they live. We agree with the remark of Deleuze [5] regarding the creation of a moving image: "While the light is moving, the movement-image and the image light are two sides of the same appearance… ". And naturally, to make a movie, you need light, just as in the Bible, where light was created to make the world out of darkness (Genesis, 1,3). Light Beings (Figure 3) are the first components of Cinema Beings. Painting Beings (2005)

5.4 Eye Beings Another important category of artificial beings in our system is Eye Beings. These are designed to "see" and "read". They see the world they are in (together with other Cinema Beings), but also see our world (with pictures from all sources) and read texts in all languages. Their role is crucial: they are the ones that establish communication between the worlds (real and artificial), and between Cinema Beings. They transmit images, and seek to interpret words and Light Beings (2003) texts. Because they are relatively primitive artificial beings, they use Google Images and other search engines for images and videos. These 5.3 Painting Beings search engines enable them to transcribe any text The first "creative creatures" of the world are in visual form. These images and videos, which Painting Beings [16][17]. These artificial beings are later forwarded to Painting Beings and other are self-organized to create vivid paintings in Cinema Beings, will be the subject of the first motion (Paint) and also to try to interpret our matter images [6] for films produced by the world (through a webcam, or images, videos, etc.) system. to propose an adaptive artificial view. Initially, they use rules of cellular automaton or Boids 5.5 Sound Beings systems (as described by Craig Reynolds in [18]) All sounds and music of Cinema Beings are We may compare them with certain works of created by Sound Beings. Their main activity is to Tara Krause [19] who created the painting from listen (again via the Internet, with search engines cellular automaton, but also the amazing and such as Google Music Search) and also to play interesting definition of "Arton" as an elementary sounds on a computer and produce sounds and particle of Art, proposed by Acerbi [20]. music following both chaotic and organized rules. Thus, Painting Beings (see Figure 4) suggest a world of artificial painting in motion, and echo the Sound Beings use fractal algorithms to generate words of Jean Epstein : their own music and are based on the Fibonacci "All surfaces divide, truncate, decompose, sequence, following rules of interesting rhythmic break, as we imagine they are in the eye with a construction, as described by Fauvel, Flood and thousand facets of the insect. Descriptive geometry Wilson [22]. whose canvas is the plane of the end. Instead of undergoing the prospect, it splits the painter enters into it ... "[21].

123 5.6 Cut Beings All Cinema Beings have certain parameters: Cut Beings are artificial creatures whose task is color, position in space, rotation, light intensity to make the montage of the film. They are the (for the Light Beings), length and direction (for timers of the film. They can work in real time, Eye Beings), duration (for Sound Beings and Cut pointing out the correct Eyes Being and putting Beings) etc. them to work. They can also make a metric The total number of parameters in the ecosystem montage for one or many created films after is enormous. producing a series of numbers in time and collaboration with Eyes Beings. The first rules As we cannot use all these parameters as part of they use are inspired by Eisenstein’s theory [23] an engine for evolutionary creation, a factor 'P' has about metric montage, which we can see also in been determined, which is a limited number of his film ‘October’ and in some Peter Kubelka parameters used for each Being in the evaluation films. This type of montage rule is based on the function, and which is constantly changing over length of plans in relation to the total length of the time. film, which is particularly interesting here (as P is just a float number, used to compute explained below). relations between the Beings’s parameters. For Cinema Beings "living" in close relationships, their parameters (with the limit of 6 Implementation of the Ecosystem the P factor), are associated either in pairs or in The full Ecosystem was built with open source sets (up to 10 parameters). These settings are software tools. The main ones were Blender (for stored as a list. The system works upon a genetic the establishment of the artificial world in 3D) and algorithm, based on relations between Beings, not Python for programming the system itself. on the Beings themselves. All Cinema Beings are self-organized and can Here is the main algorithm for the System: operate independently. Constantly changing and Begin World evolving, the artificial world cinema remains controllable by the user, who can impose a number Initialise Population: of rules, including scenarii instructions. Initialise Painting Beings The problem of the storytelling, which can be Initialise Light Beings regarded as fundamental in classical cinema, is Initialise Eye Beings here strongly deflected by Cinema Beings (whose Initialise Sound Beings understanding is not easy), and is used here more as the starting frame of the real thread rather than a Initialise Cut Beings hypothetical history. Begin Simulation The important thing to understand here is that Generation = 0 Cinema Artificial Beings are a creative Eye Beings Reading (Web, webcam, etc.) community, where every being lives in complex Image retrieval from a list of keywords relationships with others, but where none of them (scenario) has the role of director. Sound Beings Listening (Web, webcam, etc.) 6.1 An Evolving World Sound retrieval from the same list of keywords This system is a creator world in perpetual P = random(1,10) # P is the number of parameters evolution. For this, it works on the basis of a used for the fitness function genetic algorithm that tries to meet certain criteria throughout the process of making the film. Relation Beings Selection (randomly) Evaluate Population Cinema Beings lives like autonomous beings. Operating in conjunction with our real world While 1: through images, videos and sounds, they must Relation Beings Selection (with roulette apply rules to self-organize and produce wheel) interpretations of what they see. Relation Beings Mutation

124 Relation Beings Crossover LightBeing,intensity = 2.2 P = random(1,10) PaintingBeing,x = 3.8 Evaluate Population We get a new aesthetic ratio equal to 1.727, End While. which is better, and so on. End Simulation

6.2 Rules of Divine Proportion The selection criteria of fitness have become natural measurement units in very classical art, for example the values of divine proportion, as described by Pacioli [24]. See also the excellent work by Markowski about The Golden Ratio [30]. Four basic rules were retained; three of these work with groups of two parameters, and one with from 3 to 10 parameters. Cinema Beings (2012)

Rule 1 uses the golden ratio as a direct Other rules work on the same aesthetic relationship between two parameters, namely: principle: Golden Ratio, called Phi (φ), which is equal to: Rule 2 uses the opposite of Phi: (1 + √5) / 2 = 1.6180339... (Rule Factor 1) 1 / φ = 1 / 1.6180339...= 0.6180339....(Rule Thus, each Cinema Being may be related to Factor 2) another Cinema Being via the Divine Proportion, Cinema Beings also use the Golden Angle, the but this usually happens in a very complex way value of which is calculated as follows: (and is therefore invisible to the eye, but may not be to the sensitive perception of the subconscious) 360 * (1 - 1/ φ) = 137.51°, or about 2.399963 (Figure 5). radians. (Rule Factor 3). (For example, the position of a Painting Being Finally, Rule 4 is intended to link more than two may be in the ratio of Phi with respect to the light parameters (up to 10 here, but we could increase intensity of a Light Being, etc.).. that number), and requires the associated parameters to follow a Fibonacci series (this is Consider a Light Being with an intensity particularly widely used in the metric montage parameter set to 1.0, and a Painting Being with an durations for Cut Beings). x position to 3.4, (these two parameters are randomly chosen by the system). 0, 1 ,1 , 2, 3 , 5 ,8 ,13, 21, 34, etc... (Rule Factor 4) This comes up: Of course, other aesthetic rules may be used, and LightBeing,intensity = 1.0 there are plans to add a few, including some PaintingBeing.x = 3.4 directly from film theory, described by Aumont [25] Deleuze [5], Epstein [21] Eisenstein [23]. The aesthetic ratio between the two is : 3.4/1.0 = 3.4 The Genetic Algorithm is then run (for these 7 Experimentations two parameters but also for all the others). Fitness is based upon the Golden Ratio (Rule 1) for these The Cinema Beings first experiment was to test two parameters. a simplified version of the system, named Galaboids, which was mainly focused on the F = 1,618 behaviours of Reynolds' Boids applied to Painting The next step produces n children for the Beings. (Presented at GECCO 2010). parameters (here we have n = 50). Then, a first complete film, named "Galaone" The best couple is now: was released in 2011. This is the first realization of

125 fully autonomous Cinema Beings, a film created achievement of production and great patterns of by an ecosystem of artificial beings. mind-set for cinema. Then, a new project is based upon the It is also an open system, based on the principles Silmarillion by J.R.R. Tolkien. The idea is to give of Open Source software, which will allow many the novel to Eye Beings, and also some knowledge future improvements. It is an illustration of the found on the Silmarillion encyclopedia thought of Epstein [21] that images are thinking, (see at: and this was strongly echoed by Schefer [26]: http://home.comcast.net/~mithrandircq/Sil_encycl opedia.htm). "The cinema has a reality of matter and equipment specific to show, sometimes the movement of figures, sometimes the motion of matter of which these figures are imagined to be composed."

References

[1] Sims, Karl. Evolving Virtual Creatures. Computer Graphics (Siggraph '94 Proceedings), July 1994, pp.15-22 [2] C.Sommerer, L.Mignonneau and G.Stocker, Christa Sommerer and Laurent Mignonneau– Interactive Art Research (Vienna/New York: Springer Verlag, 2009) [3] McCormack, Jon. Impossible Nature. Silmarillion Project (2012) Published by the Australian Centre for the Moving Image. (2004). Hundreds of Phi values, its opposite, the Golden [4] Bergson, Henri. Matter and Memory. Angle and partial Fibonacci sequences were used Forgotten Books (April 15, 2010) to calculate the resulting images of the project. [5] Deleuze, Gilles. Cinema I. Continuum Of course, it is very difficult today to analyze (March 19, 2005) how these results work in relation to artistic [6] Aumont, Jacques. Matière d'images. theories. Because it is a very experimental system, (Editions de la Différence. 2009) we need to conduct many tests and search with it [7] Metz, Christian, Langage et Cinéma, before achieving significant results. Larousse. Paris 1971. One of the big questions is: [8] Bonet, Eugeni. Calculated Cinema. Verbindingen / Jonctions-5. What visual effects can we expect with this http://archive.constantvzw.org/events/vj5/calcula system? tedLF.html [9] Le Grice, Malcolm. Experimental Cinema in 8 Conclusion the Digital Age. (Ed. BFI Publishing .2002). Cinema Beings result in an artificial creator [10] Lattanzi, Barbara. Cinema Software and world, for creating experimental films, and putting Audience as instrument. New Media Caucus, into practice the aesthetic theories of film, (98th College Art Association Conference, painting, and arts in general. Feb.11, 2010) Films produced by this ecosystem are chaotic [11] Andrews, Jim. About dbCinema Flash and seem abstract. However, they are created with Brushes. (2009 essay). precise rules, and scenarios that can be. In addition http://vispo.com/writings/index.htm to being able to produce art films, the system is a [12] Manovich, Lev. Soft Cinema: Navigating the platform for interesting aesthetic experimentation. Database. (The MIT Press, 2005) It is also intended to serve as a link between the

126 [13] Lobb, Iain. Generative Cinema and Dialogue. University of Plymouth. (2003). [14] Whitelaw, Mitchell. Metacreation. Art and Artificial Life. MIT Press (2004) [15] Lioret, Alain. Emergence of L-Cellular Morphogenesis. Generative Art 2004. Milan. [16] Lioret, Alain. Being Paintings. Siggraph 2005. [17] Lioret, Alain. Galatema: A Framework for Generative Cinema. Generative Art 2010. Milan. [18] Reynolds, Craig. Flocks, Herds, and Schools: A Distributed Behavioral Model, in Computer Graphics, (SIGGRAPH '87 Conference Proceedings) [19] Krause, Tara. Insights Into a Cellular Automata Model of Abstract Painting.. Journal of Cellular Automata. (2006). [20] Acerbi, Libero. The Wave Particle of Art. Generative Art (Milano, 2010). [21] Epstein, Jean. Ecrits sur le Cinéma. Ed. Lherminier (2005) [22] Fauvel, J., Flood, R., Wilson, R. Music and Mathematics. From Pythagoras to Fractals. Oxford University Press (2003) [23] Eisenstein, Sergei. Selected Works: Towards a Theory of Montage. Ed. I.B.Tauris . 2010 [24] Pacioli, Luca. Divine Proportion. Ed Abaris Books. 2010 [25] Aumont, Jacques. Les théories des cinéastes. Armand Colin (30 juin 2005) [26] Schefer, Jean-Louis. Du monde et du mouvement des images. Cahiers du Cinema Livres (2 octobre 1997) [27] Davenport, Glorianna, Improvisational Media Fabrics: Take One . 2003 [28] Bocconi, Stefano, Nack, Franck, Hardman, Lynda. Using Rhetorical annotations for generating video documentaries. ICME 2005. [29] Pask, Gordon, Interactions of Actors (IA), Theory and Some Applications. 1993 [30] Markowsky, G., Book review: The Golden Ratio by Mario Livio, Notices of the AMS, Vol. 52, No. 3, March 2005.

127

Regina Freyman

ITESM campus Toluca, México

[email protected]

Facebook Chronicles

Abstract: empty space will soon be filled with nice messages” In this paper I explore my personal experience in Social Networks, Facebook, in particular. In this Five years later, having added 1178 friends, my narrative exercise I try to explore the way in situation is great, I went from the Freudian oral which this media has become transcendental in stage (here is called the chat stage), to the our day-to-day experience, and how cyberspace voyeuristic video era. I must confess that I has become an extension to what we call reality. spied friends and even enemies. I discovered that I became shy, so I preferred to communicate in intimate inboxes. I have collected contacts, pictures and quotes. Sometimes I have the feeling that Since I am a teacher, I also use this social space to this (internet and social answer my students' questions and to networks) answers to the upload notes. In the college where I work there nostalgic old idea of God, are tools like Blackboard, but nowadays who who watched everyone wants to spend his free time at and that nothing escaped school? Facebook is the new public square, his gaze and scrutiny. my evilness makes me post their absences to class and their grades, or distract them from J their enjoyment when I post the instructions for a the next project. v Before providing youi with the predictions for my cyber future and the equadrature that the search engines show (I happenr to believe that Google is my ascendant and Yahoo my regent house), I will tell you my story in thisM media. a My first impression r was nostalgic. I found a friend from my childhoodí , but then I got my first network friend, Marcoa Levario, who I had met so many years befores via e-mail and who made me part of his media magazine etcetera. The first I was born for Facebook on April 9th, 2008 at time I listened to him was during an interview on 6:53 pm, so I will ask an astrologer to tell me the TV and I was captivated, so I decided to write about my virtual future. This social network is a to him and he wrote back. As of that day, he is time machine, you can look for old friends, upload purely my virtual friend. We met each other pictures from the prior party, settle appointments thanks to the cyber surfing, and despite we never for the next day, but I know very little about my see each other, in fact we are the Buzz and Woody future here. I am young regarding cyberspace of the Google era. issues; I am barely five years old, at most, fifteen if we consider my first e-mail address, but all this Thousands of students have come back to me with was during the digital era, before having a wall grateful messages; I also managed to find and pictures that foreshadow my existence. My my elementary school teacher Lety and my first what are you thinking? Was this gentle fellows from that stage of my life. Since I was message: “This wall sounds lonely I hope mi moved by all these facts, I decided to write my

129 Facebook chronicles. I thought that if the posts, new faces awake to tell their story. Every children of Narnia escape through a closet I could morning I wake up happy, I log on to see who is do the same through the screen. I express there, to call for e-assistance, to say good my fascination in these terms: morning , to congratulate friends for their birthday and to arrange future meetings. I From Facebook I recover past life scenes, recover lost memories to decorate my wall. I characters with whom I skipped a hug, or upload photos to share my story and rescue some postponed a claim, we owned a kiss or we other pictures that several people had of my own. simply clarify an uncertain impression that today I write these chronicles of facebook hoping to is satisfied. I discreetly struggle to elucidate the reconstruct with words the faces that strong passion that the new media makes me feel; time vanished from my mind. unlike some critics, I think that the Internet does humanize us. There was a time when my self- esteem grew presumptuous: Recently, I had a cyber-meeting with a friend from my childhood. A friendship request Lately, from my dissertation about facebook, I from a certain Mauricio woke me up one morning, have found myself to be expensive, and what is he appeared as a friend of my brother and I more, there is a group of people competing to buy accepted him thanks to the reference, but I me. I still do not understand what this application really did not remember him until he sent me called Owned! is about and it worries me. I a message. I never thought it could trigger so hate being auctioned, one day I belong to a many feelings. He was someone forgotten in a crazy blonde woman, another, to one of my gully of my memory. When we were kids we used students. They want to control me! I'm not a to play every day. It was a sort doll so they can toy me. Of course, it would be of Peter Pan commanding lost children. In spite of worse if my price was lower or no one would like the distance, the network brought us back together to buy me at all. The pale division between being once more, and all of a sudden, we were sharing objects and subjects is unstable, that is why men life, and telling secrets again. and women get confused. Our material goods condition us and we do the same to them; Going on with the chronicles, I found you can enter to someone’s home, walk among his the Facebook page of Montessori de la belongings as if you were an inspector or Montaña, Lety’s school, my former elementary a keen reader able to decipher the past, the school teacher. Now, she lives in Cuernavaca and present, and their best kept secrets. The I decided to visit her, so I asked for permission at objects tell us about our needs, about our work to sleep at my former teacher’s ideology, they prepare the roads to launch our place. My first surprise was to share the bed with projects. Things are the double of our lovers, the Bob Dylan´s and Garcia Lorca´s dolls… what an ones we keep in our wallets or under our ecstasies! It made my night a delight. Who can pillows, souvenirs of melancholy, the roots of our sleep with two of his idols without having to history. The ties that bind us to our things are pay the consequences of a ménage a trois made in stronger than we imagine, we use them to incite heaven? The next morning, Lety made my lunch, admiration or to provoke envy, a feeling that some another dream come true. How many children people believe gives them success. I am auctioned would like to sleep at their teacher’s place, then in a game, but many people are got her to make their lunch and arrive at school actually kidnapped in my country to be traded in holding her hand? an infamous market where money, the symbol par excellence, is worth more than life itself. Let’s move now to my friends from high school who updated their profile on my Our bits and pieces are full of meaning; inside of new virtual life: them reside living premonitions, encapsulated memories. I wonder if my soul has not escaped It is funny to periodically meet friends from the letter by letter and bit by bit through this device, past and get up-to-date in our lives. The bearings my computer has become a branch of my being, it take us away but we always meet again over a stores my confessions, it is the key tool for all my cup of coffee. I cannot blame facebook for work, my contact with far away people, the trunk what we did before, but thanks to the pictures, and

130 of the memories that keeps snapshots devil to keep forever his fair and young and movies of my life. It saves my favorite appearance. Facebook (and all social networks) music and ideas, those I fear to forget. The are the new mirror of us mortals, a kind network to the one you connect is an infallible of pink synopsis of our lives, where the hidden tracker and in just seconds it can locate my Narcissus is hopelessly shown. From there we can physical whereabouts. I also know that it is extract some profiles that can be summarized as intrusive, that it is keeping track of all the sites I follows: navigate, and all my information is crime and alibi of my relationships, hobbies The users who like to compete for the larger and habits. The network is like money, an collection of friends. ambiguous symbol, a way of contact and trade, a Fans of Happy Family, those who exhibit their mean to make war or to promote peace perfect clan. proposals. A new impostor has bought me again; the notification arrives to my cell phone. I The “I got money” character, that loves to upload demand to be explained why I am being the souvenirs of his trips and extravagant life purchased and sold, I do not want to end up as an mementos. avatar in the caverns of cyberspace. The insecure that feel they could be kidnapped, so For sure, its space is practically empty. my Facebook shows my absurd assumption of myself as a literature intellectual and I sort quotes The forever-young who loves to imply “Look at like a gangster that sorts bullets to his enemies, or me; I look well despite the time”. the rich man who gives everyone tips: Or the guy that feels to be Spiderman because he “Mirrors are abominable because they increase the spends all day in the web and reports all its number of men”, said Jorge Luis Borges, but what whereabouts. would he say today if he learnt about Facebook?, he would probably think it is the worst aberration, All of the referred stereotypes are categories of its proliferation power makes it a cubic mirror and the same loneliness and vanity. Perfect reflex of a its manipulative capacity turns it into a society that fears its present, that attempts to find compelling portrait capable of projecting the itself in the past, and that hopes for the future image that any subject wants to feed as his own projects that always promise a better life. It is the shadow. The famous witch in Snow White, with ideal media of a speedy epoch which does not a different attitude, loved her mirror until have time for physical contact and turns a screen he betrayed her; it gave her an undesirable picture, into the new meeting point. At least, it is a kind incapable for lying. Like every bad lover, it could gallery that allows us to edit life, to collect not keep it a secret the youth supremacy of Snow friends, and to share the perfect album in search of White. How many troubles would have been a bit of admiration from others, and to exhibit saved if the bland mirror had replied again: You, ourselves; after all, life turned into images looks my queen, are the fairest of them all?! But better, and allows us to recover that moment that except for that time, the mirror, as a we could not taste because we had our sight fixed faithful servant, helped her reclaim her beauty and on the event that is going to become the post of gifts; it also served as a locator for lost friends the following day. or stepdaughters, an antecedent of the GPS without a doubt. It was even a faithful Thanks to an old friend recovered from my social partner to talk to in times of loneliness. Another site, I went to a Cold Play’s concert and I friend of mirrors, the oldest of them all, was the could see my idol Paul McCartney, at these catatonic Narcissus, who spent his life watching shows I discovered that the new generations are: his double in the waters of a benevolent lake Three-eyed beings, two on their face and the other that gave him back the picture of his virtues. This one is the cell phone’s camera which records and character as a germ of later fiction is presented feeds their social network with portraits from as the icon for the sin of vanity, and with reality. Nowadays, without that evidence, nobody Oscar Wild's fable it reached diabolical excesses. can be sure that life is actually going on. It seems The protagonist, Dorian Grey, sold his soul to the

131 that we need the pictures to be geographically flower that slowly opens to you, you give me your placed, and to give testimonies of our emotions; secret flower back and we grow together into an these images materialize our being, we have one emotional maze that palpitates in foot set on Earth, another on the Internet. the same heart of our feelings, I´ll give you a word! Since I was born for Facebook and Twitter, the Earth quaked and disasters were present, then I Social networks have become dating sites, learned that: however, it is not easy to test whether they are more or less helpful than those in real life. We Thousands of people risk their know that Facebook is very popular among those lives by upgrading the drama and ordinary people seeking greater certainty than with a blind become journalists overnight. The contingency date. The network has its romantic pathologies: and evacuation plans have to consider dead the Cyberia and homogamy, the first one is to time that will be lost when portraying and transcend one's generation and find younger or narrating the disaster… Will there be people older partners. There are no ages for love! The who, even in the midst of an orgasm, try point is to hit upon sharing views no matter the to measure its intensity? year of birth; the second is the tendency to gather people who are alike. We look for Love is another focused area in my social simultaneous passions and parallel sincerities; network: the networks make us feel that no matter what Two lovers are separated; the screen approaches happens on the web stays on the web. them once more. Their words touch and caress; Going on with another point, I must admit I was a they make love one another with the political activist. I recovered the notes of the keyboard. They shortened the distance and they victims from the drug dealing war, but are together again, their souls smile, sing, my wall became bloody and I decided that dance and write, subdued they throw like my Facebook was an escape and not a tool for confetti, here and there, love phrases, song’s complaining: paragraphs, expansive joy winks that raises comments: "I like you", "I will copy the In my country, Mexico, violence is phrase", pleasure spreads. normalized, the media is sensationalized, the fiction is fictionalized and death no longer scares A word shall be enough to heal me, seduce anyone. I do not want to watch, I tell myself every me, excite me or embitter my day. Twitter and day when the notes and images of a war I do not Facebook enhance the word (of course, also the believe in display fragments of dead men and images) are an intravenous injection that travels women abducted by misfortune, “They are the bad directly into the bloodstream, lodging like a virus, guys”, my little girl says to me to spear my an expansive idea that grows and spreads: tell me sadness but I know it's not true, because the your wishes, the darkest ones, those who you story of my country is not about wonders, and the would not dare to mention face to face and out distinction between good and loud. Are you tired of carrying that daily person, evil disappeared long ago, when we still the rusty one, the one we've built since childhood, had black-and-white TV... I'm tired of the one that has dents of old battles? Do not hearing raves about the achievements worry, we built a new profile. Our life in of Colombia, the luck of Brazil or the the network allows us to be reissued, to reinvent salvation that will come from outside. I'm tired of us, to open new living spaces, is brewing another believing but I refuse to surrender. I know that possibility of being someone else or continuing from now on I will bear a name and tell the story being yourself and perhaps, the essential of the victims, I will use all the means that an nature is unavoidable, but we get the luxury ordinary Mexican has to express herself: my of walking possibilities that we had left on the social network... shelf. Networks are the antidotes to every-day boredom and the new scenery to feel My sister, my daughters and I have played to new emotions. We strive to understand our write stories of princesses among posts: feelings and others’ feelings; I give you my words in exchange of yours, my feelings are a

132 "The princess decided to definitely go away and a homeless woman who perfectly captures my she packed: her red shoes to dance until she lately appearance, she said that that day she would dies; glass slippers to trace; red hood to run bring the stuff for me to make quesadillas so we with wolves; apples of discord to sow anxiety, the could gather some money to pay the hospital bills. helmet of Athena to avoid headers. She would As of that day an entire story was born, with leave the sword stuck in the stone; the head of her several added people and the truth is that mother that, when looked at, petrifies her; my laughter was immodest; I feared I would the useless pumpkin that breaks the spell at 12:00, be jogged from there. It is interesting how the treacherous mirror that eventually finds a this tool is used today. My sister got a prayer more beautiful woman, and as an ecologist she chain around but I hate the religious roll would recycle “Once upon a time…” as many and public mourning. It is not the same for me as times as necessary." an exhibitionist; I prefer to expose phrases, to send messages to my students, or even silly jokes, "Andre I thought that thing about but that stuff of "poor of us, pray for my mom", your mother was temporary but I think she is no way! To tell you the truth, I was furious. My smoking peyote or other substance, now she wants stupid sister stole my I-Pad for a moment to be Peter Pan, I have really tried, you know it, to and together with it my identity on Facebook, and deal with her things, but now she wants us to posted: "I've always been a combination of write 420 words x post, and as you see I have Marilyn Monroe, Hugh Hefner and Octavio Paz." children and I can’t devote myself to your As you can imagine, the post unleashed a rain mother’s follies." of comments until I noticed and cleared the encroachment. Intensive therapy enhances "Mirror, mirror on the screen tell me who is the affects, dramas, emotions and fairest of all facebookers, you're the most even stupidity. Finally at home, I put a few drops beautiful Andre but there is an old fool who is of Channel Number Five, my silk robe and I called Patiki that competes with you despite she is began to rewrite the Labyrinth of Solitude. in her forties." I do not like video games or their imitators, but "I'm sorry to announce that I may leave the game I’ve found a new way of family life, I interact since I've noticed some discrimination or with my father who uses his facebook just to xenophobia on your behalf, you play "The Wheel of Fortune" or never pick Pocahontas or the Chinese Mullan. On "Jeopardy". My social network the other hand I am invited to a site in which they is my promoter and I call my friends to read are playing to be Pokemones and they let me play my texts: as the evolved Pikachu." "Read me, read me, I talk about sex and it sells, "Dear Prince I know you are afraid of me because read me because you are my friends and my I am just the perfect slipper that fits your students, and because I am a good person, and dreams. I´m your princess my Cinderello.” CARAMBA READ ME..."

"Dear evil sister, you're not a princess, in Sometimes, I suspect that I am given some "likes" cyberspace and in real life, you will be known just out of pity or politeness; of course there are as the Iron Lady for your latent coldness some more generous students who quote me to and evil." show that they did their homework. I also use Facebook to locate my elusive brother, or to make Yours truly, a party over the Oscar Academy Awards night. Patricia the nice sister Good Heart and soul That movie night I tried to entertain my guests We have also cried and laughed of with memorable scenes and quotes of the over our family tragedies, Facebook became our nominee: shelter from an intensive care room: "I see the Oscars live with Marz and Leonardo, I spent much of that night telling stories, from with Patricia, Eduardo, Aurora, Lili, Sebastian and the intensive therapy room with my sister on many more, thanks to this network with which my Facebook. Patricia posted the picture of generation never ceases to get amazed. What a

133 great Oscar Night! " production, management, transnational distribution of goods and services, skilled labor, Internet with search engines, social networks, science and technology, media. Alternatively, emails and attachments of remote communication Nicholas A. Christakis and James H. Fowler, (camera, chat, Skype) binds souls. In that sense, considered networks to be any group that goes modern cell phones, which also group this from a couple, to spaces such as Facebook or technology and have the advantage of being World of War craft. The authors of “Connected” portable, are means of contact too. Thanks to the analyzed information about the happiness of 4739 creative skills involving the use of a computer, persons and its connections with thousands of this becomes a scenario that allows artistic people (about 50,000 social links) they concluded creativity and facilitates the production of that when people changed from unhappy to happy, expression. Men and women of today spend many the social network contacts did it as well. So be hours reinforcing individuality, much of this by careful what you say, show, or post (by using a using sophisticated tools put to our service that neologism) in the network. Online, we are not allow us to create from a phrase to a movie, it is anymore individual entities and gradually we itself a wonder; but it also gives us the catwalk to become networks, microorganisms that resume display our personality in front of millions of our individual strengths. peers and then, it has become an almost necessary validation.

The neuroeconomist Paul Zak tells us that Social networks are here to extend our lives, they networks delight us almost like an orgasm and work slowly and increase our range of reduce heart attacks, he found that oxytocin levels communication, we recover lost friends or distant rise after a session of Twitter and stress hormones, relatives, we duplicate the work schedule (it is cortisol and ACTH decrease. Social networks common to see families apparently together, but at unleash powerful physical changes in the the same time very far away from each other, at a chemistry of our body, hence its addictive power. restaurant, for example, without talking, To our brains, interacting with people in the answering calls or messages from other people, or networks is so "real" like being with them solving tasks or setting meetings). Why wouldn’t physically. We have a narrative brain, we process erotic and romantic possibilities be also extended? information in stories, and so we are fascinated All I do is related to stories; I guess that when I with this public diary. was a kid, somebody read me a story that On the other hand, we like to build relationships: contained a spell. We are the stories we tell friendships, family, work, etc. Since the ourselves, I always say that and I believe it. The eighteenth century, sociologists study the power new technologies relive old words; revive of community ties in various groups, such as emotions just to remind us that, in the center of tribes, families, and, labor, religious, and stories, projects, links, past and future stories, the educational systems. In the fifties, the term "social creative word reigns. We are our words, those that network" was defined as a group of people who now shine like neon in multiple, fast and fugitive maintained ties with defined cultural traits. In the showcases, social networks give us the late twentieth century, with the appearance of opportunity to rewrite, it is always the possibility virtual communities, human beings can meet each to reflect, to lie, to hide what we don’t want to be, other in spite of distance, favoring traits of affinity but at the same time, to become what we actually on qualities of identity, we have friends because want to be. They are the diary that we always we sing the same tunes, and we are excited hide, but dream for other people to find and about the same plots, we are fans of the same discover the best version of ourselves, the edited painters, no matter the age or nationality, our version of us that we do like. That's why I do not tastes make us match. Manuel Castells said that like Twitter, it is a pretentious social network, social networks are part of the "network society", where users feel the obligation of being intelligent a structure composed of "digital communication in 3 words, in Facebook we are exhibitionists, technologies and electronic-based information" child-likes, sentimental, romantic, vain and that affect human life in every corner of the planet attention-seekers, the Facebook showcase, the through global webs: financial markets, post-modern times biography that makes us

134 music, image, movie, quote, word, or translate the "On facebook we never die." complexity of our emotions in signs that send us back to the pictogram:

Q<= :/ !…ʃʘᴥʘʅ : ( 1000000 \o/ \o/ \o/ \o/\o/

Regina, bored, bought a dog , continued sad wanted to have a million of friends

---<@ --<@ --<@ @

* / c <3 ///// J

To throw her flowers and messages, she joined Facebook, day and night her heart beats happily.

)}i{(

Fluttering like a butterfly.

Now, it's time to talk about the future, I appeal to Rasmunda the videnteastrologa.com, and she tells me that my web-life looks well, I will continue receiving friend requests, I will never get old with a new application that will wipe away kilograms, spots and wrinkles. It seems to be approached long fortune inbox, chats full of joy. By serving 35 years of my life in Facebook, I will prick mi finger with a key corner and I will die, the cursor arrow will be blinking, while popping chats will try to revive me, kiss emoticons will try to awake this senile Princess, there won’t be success; my secrets, my heritage of trivial words and tricked pictures will be delivered to my daughters from iTunes in a pretty interactive device. Nine months later in a reverse childbirth, Google will erase any record of my steps over the web. In a 100 years of Earthly life, my niece Alexandra will be recognized as the first Mexican facebooker novelist and she will revive me as a Pancho Perules lover, Esthercita de la Mar’s boyfriend, the one who abandoned her on the church altar, those were the non-existent profiles that she created at eleven years old and that became an example of my narrative class to prove that new media are in favor of creation, that a child was read by many, without knowing how to spell or perfect syntax, without distinguishing narrative techniques, only with the resources of her precocious imagination, her TV-serials and fairy tales collection, but above all, because she was born almost simultaneous in reality and virtuality.

135

Architectural Ornamentation and Fabrication with Multi Agent Subhajit DAS Florina DUTT School of Design, School of Design, University of Pennsylvania, USA University of Pennsylvania, USA 777 Xinzha Road, B-1 Apt 23A, 777 Xinzha Road, B-1 Apt 23A, Shanghai, 200041, China Shanghai, 200041, China [email protected] [email protected]

Abstract inspiration to produce ornamental artifacts. After setting up the computational framework to From a historical perspective, we were extremely achieve the intended output, the next phase of the intrigued by the organizational complexity and project indulges into 3dimensional artifact integrity of traditional ornamentation styles in production with the help of state of the art rapid architecture & furniture design. One of the prototyping techniques and digital fabrication remarkable styles of interest is Rococo or "Late tools. This confirms the practical feasibility of the Baroque", which evolved in France & Italy for entire computational process and ensures the architectural structures and furniture décor. The possibility of large-scale production of intricate research project conceived from the notion of how design. Computational procedure adopted for the design computation could be a valid medium to generation of the basic framework is Multi Agent generate ornamentation in design comparable to System where in a self-organized system is the one found in traditional architectural & established comprising of thousands of furniture décor style. The research work autonomous intelligent agents. These agents predominantly studies the extent of ornamentation interact among themselves and educate each other in Rococo Style at the scale of both Architectural with their embedded behavioral traits and in this Design and Product Design. Rococo was chosen process, they end up exhibiting complex for it resembles very closely with the degree of non-linear designed pattern. These patterns are of complexity and richness of ornamentation desired value to the designer and they can calibrate the from the output of the complex computational script so formed to get desirable design output. design process. The rich grandeur and florid beauty of the motifs and the sculptural tectonics in Keywords Rococo is highly appreciated and thus, intent of Multi Agent System, Swarm Intelligence, the project was to develop a self-organized Artificial Intelligence, Design Automation, Digital computational framework, which is calibrated to Fabrication & Prototyping have the potential to form complex designed artifact. Adding to that, the project strictly adheres to the objective to create a computational methodology or script which can provide series of such designed artifact, based on designers input parameters and initial conditions. It is noteworthy to mention the objective of the research endeavor was not to simulate and imitate rococo design and pattern formation by design computing. However, our research goal was essentially to form a computational procedure with the aid of programming and scripting techniques to generate florid, graceful and ornamental design artifact in production, which might or might not resemble rococo design theme in its appearance. Rococo Fig. 1 The study of the ornamental structure and motif formation in an 18th century furniture design. style is mentioned to emphasize our source of

137 1. Aims architecture, interior design, decoration, literature, music and theatre. The Rococo was developed as a 1. To study a chosen old architectural design reaction against the grandeur, symmetry and strict style-having ornamentation as a strong design regulations of the Baroque. Rococo design feature. To draw significant inferences and characterized a more jocular, florid and graceful conclusions from the same. approach to Baroque art and architecture. The 2. To conceptualize the logical flow of the design during this era was ornate and made bold preliminary computational framework . work of creamy, pastel-like colors, asymmetrical 3. To develop the design computing solution to designs, curves and gold. Unlike superbly rich and achieve highly ornate and floral design work. extremely ornamental Baroque, the Rococo had more playful and witty artistic themes. About 4. To produce 3d physical models or prototypes interior decoration, Rococo rooms were designed from the computational model to understand the as total works of art with elegant and ornate integration of design computation with rapid furniture, small sculptures, ornamental mirrors, prototyping technologies. and tapestry complementing architecture, reliefs, 2. Rococo Design Style and wall paintings. The above-mentioned design attributes inspired us to select rococo as our theme In this research paper, we often use two terms, of research study. Consequently, the remarkable which are ornamentation and motif formation. We research results motivated us to make the use and understand the term ornament as a preliminary computational framework to obtain decorative element to embellish parts of a building such design pattern of ornate quality. Refer Fig 2. or object. We identify such delineation of Evidently, at any point of the research process, our buildings, furniture decor is dying, and this goal was not to simulate or imitate exact behavior research endeavor is partly an attempt to relive the of rococo pattern. Moreover, we intended to be beauty and grandeur of ornate and florid design surprised and awe bounded by the complex and work. At this context, we also define motif as an intriguing design work from the custom written element of a pattern or theme. It could be repeated script, in the computational framework. in a design or composition, often many times, or may just occur once in a work. Refer Fig. 1.

Fig. 3 The study above depicts the layers of curve interactions that can result into the degree of complex ornamental designs in the 18th century furniture design. The diagram outlines the classes in the computational framework and their interaction pattern to synthesize such design pattern. Fig. 2 The analysis shows the motifs in the ornamental structure of rococo showing type of curve structure evident in its geometrical construction. 3. Computational Framework The coding for the computational framework was As mentioned before, we were inspired from done in java based applet Processing. Basic Rococo as an ornamental design movement in programing technique adopted for the design olden times; we conducted historical research computing was through distributed intelligence in study of the same. Research data revealed that a complex system of multi-agency.. We were Rococo (also known as "Late Baroque"), interested to learn and develop from the originated in the early part of the 18th-century self-organized interactions/ simulation of multi Paris, France. It is notable for its artistic agent system, where the embedded artificial movement and style, sprawling vivid domains of intelligence of each agent reacts with each other the arts comprising mainly painting, sculpture,

138 based on predefined logistics. These reactions and and other neighboring agents in previous feedback procedure imparted a visual pattern of simulations and thus the system continues. self-organized complex system, having aesthetic Analogous to ant colony optimization technique qualities similar to traditional ornamentation style. these agents follow those pheromones more Refer Fig. 3. Adding to that, the algorithm is based strongly which fulfill predefined conditions like on Craig Reynolds Swarm model (Boid systems) distance threshold, angle of vision or positioning specifically developed upon the behaviors of close to certain kind of agent types. Along with Cohesion, Separation and Alignment as exhibited the ant pheromone behavior, these different by a swarm simulation. Research work classes are embedded with the following encompassed pheromone trail formation in Ant behavioral traits, which makes the simulation colonies (as exhibited in Ant Colony optimization produce obtained results; methods), by which ants lay down pheromones directing each other to resources while exploring 1. Material Class (the base material their environment. This trail obtained after laying following swarm movement) pheromones gets stronger & stronger in further simulations, essentially based on the ants who 1.1 Exhibits Basic Swarm behaviors achieve stronger solution sets in the previous 1.2 Agents aligns to self simulation. At the designer’s level, this web of 1.3 Seeks trails formed by self complex pheromone trail in 2 or 3 dimensions is 1.4. Has intelligence of range & angle of vision of aesthetic and ornamental quality, which could be used as a potential artifact of high dimension 2. Ornament Class (the ornamentation sculptural tectonics. following the material agents)

1.1 Agents aligns to self 1.2. Strongly seeks trail formed by material class. 1.3. Selectively aligns with material trails based on distance 1.4. Forms branching due to inter relation with material class

5. Motif Class (Formation of motifs on the ornamentation class based on interaction) Refer Fig 4.

1.1 Formation occurs based on time factor on ornament trails. Fig. 4 It shows study of motifs found in the ornate quality of 1.2 Form is analogous to rococo flowers or motifs. Rococo. The study revealed procedural techniques in design 1.3. Steers towards a pre assigned curve formula. computing that could be adapted to form such similar kind of 1.4 Very strong steer scale motif formation. Mathematical curve formulae and logical 1.5 Dies based on distance from branched rule sets governed the formation of motifs in the script. ornaments 1.6 Dies based on time factor, from the time of 4. Class Formation formation

The code begins with sub division of the agent 6. Motif line Class (On ornament trail systems into four different agent types, behaviors following motif formation) of each of them are defined by their respective classes. Each of the agents in the following classes 1.1 Formation based on time factor portrays ant’s behavior and they release 1.2 Forms after motifs pheromone at any change in their position vector. 1.3 Steers to a different curve equation. In the subsequent simulation, these agents then 1.4 Steers with less strong factor than motifs tend to follow the pheromones released by them

139 1.5 Dies after some time motif formation. 1.6 Dies based on distance from branches. 2. Steer factor calibrated to show different motif formation The above structural organization was the basic 3. Motif was produced after certain count hierarchy of the code at the initial stage, which we of ornament trail agents. further developed to exhibit much refined pattern behavior.

5. Algorithm Development

5.1. Phase-1 The above organization of the code started to show pattern formations but the simulation result was not coherent and appeared to be uncontrolled haphazard pheromone trail formations. The pattern so formed was characterized by multiple pheromones branching within a threshold distance from neighboring agent. However, by the end of Fig. 6 Shows the flow diagram of the logical rule sets among the simulation, the branching of the ornament trail the classes and their interaction in the program code at and motif died into a single line. Moreover, it phase-2. started from a random incoherent state and took some time to achieve desirable results. Refer Fig. 5.

Fig. 5 Shows the flow diagram of the logical rule sets among the classes and their interaction in the program code at phase-1.

5,2. Phase-2 This phase involved addition of a deviate function into the material class, which primarily created a sense of controlled randomness in swarm behavior of all agents. This deviation of the material class exhibited unique output and design pattern. This Fig. 7 The diagram above shows the curve branching and function was initialized based on count factor of pheromone trail calibrating techniques adopted in phase-2 to number of material class agents distributed. Motif create substantial improvement in pattern making from the and Ornament class were modified as below: code output in phase-1.

1. Different curve equations tested for

140 The result shows lot more coherent interaction resulted into similar coherent and branching (closer to desirable design complexity), controlled design ornamentation in 3-dimension which stays for long and does not form into a line space. Refer Fig. 9 & 10. as before, yet there is extreme randomness and incoherence at the beginning. Branching by the end is not up to the desirable point. Refer Fig. 6, 7 ,8 & 10.

5,3. Phase-3 In this phase, the material trail was not updated as it showed appreciable results after the inclusion of deviate function for randomness. However, the ornament trail now is produced after periodic production of the material trail agents. Therefore, in this phase ornament trail agents was no longer produced randomly and thus the results obtained were more legible, coherent and controllable. Besides, both motif and motif line class were equipped with “Die” function, which specifically, calibrated the agents to die out of the simulation after predesigned time interval after going through certain number of interaction with other agent types near its threshold distance. These very important code changes produced commendable results in the form of detailed ornamentation as we see in traditional design ornamentation in patterns and motifs. Refer Fig 9.

Fig. 9 The diagram explains final code logic with Fig. 8 Diagram showing motif formation rules in necessary changes in phase-3. This logical rule phase -2. sets imparted calibrated and controlled design pattern work with desirable aesthetical qualities. 6. 3D formation & Digital Fabrication The 2 dimensional pattern work obtained above is the result after phase-3 showing branches and Once the system is established and the coded motif formation. The script was calibrated to give simulation produced desired results, the 2 different results at different parameters of input. dimensional frameworks was converted to 3Dimensional setup with the addition of z- vector Eventually, another program code was written to and z coordinates on each of the agents in all of form iso-surfacing between the 3D point clouds their respective classes. This time their 3 obtained from the 3D ornamentation pattern from dimensional simulations of reactions and the simulation results. The iso surfaced mesh was

141 exported as an .obj file and imported in Geomagic computing which is the real achievement of the platform where it was fine-tuned with further research endeavor. Further calibrations and subtle mesh optimization and accuracy. The resultant modifications on the script and program code can mesh obtained was 3d printed directly from bring about infinite number of such kind of crafts Geomagic interface. The workflow from digital to each type having distinct identity, aesthetics, and physical model was from processing original complexity at the level of geometrical formation. pattern formation script to custom written iso The digital production of these at a commercial surfacing program. At the next level, the workflow scale has the potential to bring back old rich was from exporting the iso surface mesh object grandeur of ornate and florid design works at the from processing and fine-tuned in Geomagic scale of architectural design and furniture décor. interface, finally to be 3d printed as a digital craft.

Fig. 11 The 3d digital artifact obtained from the point cloud in processing. They were processed in geomagic for 3d printing. Fig. 10 The 3d printed artifact from the script at phase-2 with limited branching and motif formation. At this point, it would be notable to add that the scripted output and 3d printed artifacts, does not 7. Conclusions imitate the formation of rococo or baroque design works, but presents a new paradigm of genre of The advanced computational coding and digital digital crafts, which is novel and unique in its own fabrication tools allowed us to produce ornate geometrical formation. The multi agent system artifacts of desired scale, precision and proportion. adopted for the process, delineates a novel The procedure followed clearly unravels a novel paradigm of its application, which is to produce paradigm of design endeavor wherein very complex digital artifacts by pattern formation from complex and intricate design patterns can be the movement of the individual autonomous computationally studied, analyzed and reinvented. agents. Application in this new field of inquiry by The research project not only facilitates a genre of Multi agency is innovative and we observed ornate digital crafts, but also at the same time further work in this direction, would definitely documents the complex computational framework display enthralling results. The digital crafts so adapted to synthesize such kind of design work. produced can be successfully applied at all the We firmly believe it is the process in the design scale of furniture décor, building fenestration

142 components or ornamental units, sculpture design Bio-Inspired Artificial Intelligence: Theories, and so on. Refer Fig. 11. Methods, and Technologies (Intelligent Robotics and Autonomous Agents series)

7. Eugene Rouyer, French Architectural Ornament: From Versailles, Fontainebleau and Other Palaces (Dover Architecture)

8. Farshid Moussavi (Author, Editor), Michael Kubo (Editor), The Function of Ornament

9. Ira Greenburg, Processing

10. Jacques Stella, Baroque Ornament CD-ROM and Book (Dover Electronic Clip Art)

11. Jaime S. Sichman (Editor), Rosaria Conte (Editor), Nigel Gilbert (Editor), Multi-Agent Systems and Agent-Based Simulation: First International Workshop, MABS '98, Paris, France, July 4-6, 1998, Proceedings (Lecture Notes in Artificial Intelligence) (v. 1534)

12. Jennifer Dawn Milam, Historical Dictionary of Rococo Art (Historical Dictionaries Of Literature And The Arts)

Fig. 12 The 3d digital artifact obtained from the point cloud in 13. Khaled Nagi, Transactional Agents: processing. They were processed in geomagic for 3d printing Towards a Robust Multi-Agent System (Lecture Notes in Computer Science), 150-205 References 14. Michael Wooldridge, An Introduction to 1. Adelinde M. Uhrmacher and Danny Multi Agent Systems - Second Edition, Weyns, Multi-Agent Systems: Simulation and Published May 2009 by John Wiley & Sons Applications (Computational Analysis, Synthesis, and Design of Dynamic Systems) 15. Nirmit Desai, Alan Liu and Michael Winikoff, Principles and Practice of 2. Brent C. Brolin, Architectural Ornament: Multi-Agent Systems: 13th International Banishment and Return Conference, PRIMA 2010, Kolkata, India, November 12-15, 2010, Revised Selected / 3. Brian Earnshaw ,An Insular Rococo Lecture Notes in Artificial Intelligence)

4. Christian Blum and Daniel Merkle, 16. Rafael H. Bordini, Mehdi Dastani and Swarm Intelligence: Introduction and Amal El Fallah Seghrouchni, Multi-Agent Applications (Natural Computing Series) Programming:: Languages, Tools and Applications (Multiagent Systems, Artificial 5. Daniel Shiffman, Learning processing Societies, and Simulated Organizations) Jun 23, 2009 6. Dario Floreano and Claudio Mattiussi,

143 17. Ronald E. Schmitt, Sullivanesque : Urban Architecture and Ornamentation

18. Russell C. Eberhart, Yuhui Shi, James Kennedy, Swarm Intelligence (The Morgan Kaufmann Series in Evolutionary Computation)

19. Scott Camazine, Self-organization in biological system ,(Princeton University Press)

Fig. 14 3d printed model showing different design options obtained from the script.

Fig. 13 Some more design options showing motif and ornamentation at different scale of precision and proportion.

144 Autonomous social Avatars

Juan Pablo BERTUZZI CiTu - Paragraphe Université Paris 8, 2 rue de la Liberté 93526 Saint-Denis, France [email protected]

Safwan CHENDEB Khaldoun ZREIK CiTu - Paragraphe CiTu - Paragraphe Université Paris 8, 2 rue de la Liberté Université Paris 8, 2 rue de la Liberté 93526 Saint-Denis, France 93526 Saint-Denis, France [email protected] [email protected]

and approaches - some of which may be inconceivable - to this mental exercise. These Abstract visions are usually attached to the purposes "Autonomous social Avatars" (AsA) is an that we may have either as individuals or as interactive social tool designed to provide an innovative experience in the sphere of digital part of a larger community. In this mental communities. This application supports the idea of process, our imagination can set on a journey integrating digital environments into physical by projecting different perspectives and spaces, contributing to the evolution of hybrid representations of our inner self into a worlds. AsA will serve as a continuum of our potential future (i.e. how, where and with daily practices, particularly in both physical and whom will we live in the next few years). It cognitive levels. In addition, we intend to generate can also project various possibilities of the original fields of analysis regarding the socio- environment that surrounds our microcosmos cultural effects of new information and (i.e. what the world in general - or our world communication technologies on our society. in particular - will be like). Science and new technology are typically Keywords Social Networks, Social Avatars, Hybrid Worlds linked to the most utopian ideas of our times. Our collective imagination has evolved to conceive devices that can transport our minds Creating social paths and bodies into places farther, higher or deeper than ever before. It has created gadgets "I thought of a labyrinth of labyrinths, that can make life easier and more of one sinuous spreading labyrinth comfortable. All in all, science and that would encompass the past and the future technology are nowadays more attached than and in some way involve the stars."1 ever before to the ways in which we achieve our life goals. Without a doubt, thinking about the future This paper lays out an ongoing study related and imagining a brand new world is a highly to the HyperUrban2 research project. In our pleasant exercise for our mind. However, each one of us may have a different set of goals 2 The "HyperUrban" project also includes a reflection on the concept of a "social city," where the means of 1 BORGES, Jorge Luis, The Garden of Forking Paths communication, socialization, learning, sharing, in "Labyrinths", New Directions, 2007, page 23. working, etc. go through different states of self-

145 eagerness to create a product that goes one an individual and that of a virtual entity), in step further in the evolution of digital real-time and within a hybridized space. AsA communities, we wish to introduce the is intended to generate a persistent feedback concept of Autonomous social Avatars between the hybrid worlds and the traditional (henceforth AsA); an interactive social tool world, especially between users and their for hybrid spaces. AsA was originally avatars. Its philosophy is based on the self- conceived as an experimental tool to show the representation of users as they actually are (or potential social relevance of a hybrid territory as they consider themselves to be.) AsA is a such as TerraDynamica3. Therefore, AsA can social application that intends to be used be applied to any simulated and physical passively or actively, adapting itself to the social space. needs and wishes of each user. Our main purpose is to create a social Before describing our experiment in detail we experiment that will enable users to interact in must underline that AsA is, above all, an real time with the actions of their digital original contribution, as it enables the representation, learn new ways of socializing optimization and combination of several and ultimately expand their connexion with technological devices towards an innovative the physical world. AsA is most significant in experience in social development. At the everyday activities, as it is intended to same time, the novel idea of AsA may give combine the physical trajectory of each user way to a brand new set of original tools and and the autonomous simulation of their avatar fields of study associated with the analysis of for the same itinerary, generating real-time the impact of new information and independent decisions by the avatars that will communication technologies on our society. offer users a new and original approach to social relationships. The academic goals of our study are as Conception and evolution follows: As previously mentioned, AsA was originally * To show the infinite number of interactions conceived as a social demonstration for a that may occur in the physical world, in the hybrid territory like TerraDynamica. In order same space-time continuum and are not the to realize the full potential of this project, we most likely to occur. set out to create an experiment that could * To enable users to follow the actions of generate different fields of socio-cultural their avatars and increase their interactions analysis. This is why it is relevant to describe within the physical world (or within the TerraDynamica within the context of our simulated world). work, so as to show the basis for the * To introduce new uses of simulations for conception of AsA. social purposes. TerraDynamica is associated with the * To provide a more active role to the online dynamic 3D representation of urban actors identity of users, with consistent feedback (crowds and traffic) within a detailed with their physical identity. environment. It is an ambitious and Eventually, AsA can be used as a different collaborative work with the support of ten way of looking at the various roles that we companies and academics that specialize in simultaneously play in society (such as that of specific domains such as artificial intelligence and behavior modeling, vehicle and traffic regulation, and where many of them are based on models, 3D environment representation and information and communication technologies (ICT). real-time visualization, gaming, 3 TerraDynamica refers to the dynamic animation of a city (in this case, the city of Paris) and complements conversational agents, web and GIS. the work of the TerraNumerica project on modelling The main goal of the project is to develop and urban digitalization, with the aim of simulating smart technologies that can provide animation "life in the city": people and their masses, their vehicles and intelligent behavior (i.e., coordinated, and their movement.

146 emotional, and motivational behavior) to simulations. The perfect scenario pedestrians and crowds, vehicles and traffic, would be a 3D simulation of Paris with multiple levels of detail, (such as TerraDynamica.) However, pedestrian/vehicle interactions and real-time the first prototype could be realized constraints, all integrated into a unique with a 2D cartographical "behavior animation engine." representation of Paris. The project also illustrates the use of this • Register the Avatar simulation of the "behavior animation engine" within different trajectory followed by the physical fields of application, such as: user in the city through real-time GPS • Urban development data. • Transportation • Create, characterize and personalize • Security & Safety the avatar corresponding to each user. • New urban services The cognitive development of the • Video gaming avatar is a key element of our research Its prime purpose is to show the versatility of purposes. the proposed solution and its potential uses in • Incorporate data from popular social different environments that could address networks. This will be the easiest way several markets. to spread AsA in a large scale. • Delegate the mission of creating new Within this context, we can describe the ideal social interactions to the avatar. At steps for the development of AsA: this point, the autonomy of the avatar • Develop an interactive social is the most important and innovative application for dynamic urban feature of AsA.

Image: Schema "Autonomous social Avatars" (JP Bertuzzi 2012)

A) The experiment begins in the physical within the simulation of the Saint Michel world. The user, equipped with a GPS device Fountain of Paris. (such as a smartphone or a touchscreen B) At the onset, the avatar will follow the device), activates AsA. This will result in his user's trajectory, but it will be completely free avatar appearing in the simulated city, in the to take a different path at any given time to same physical location as the user himself or pursue his or her interests. What is more, the herself. For example, if the user activates the avatar is predefined through a range of device at the Saint Michel Fountain of Paris, settings that will determine its affinity degree her avatar will appear at the exact same place towards the other avatars or places in the simulated world.

147 In the physical world (especially in Given a large amount of users and metropolitan cities) it is indeed possible to synchronizing the application with popular cross the path of people that we know without social networks, such as Facebook or Twitter, even noticing. AsA will be able to recognize the data can be combined to generate a social these people (for example, by retrieving them graph of affinity (with friends or friends in as "friends" on different social networks) and common, with people who share rare will offer users the possibility of approaching interests, etc.) that is not easy to generate in them. AsA also has the autonomy of the traditional world. Hence, the way of retrieving other people in the physical world connecting with these new people with shared that may have a high degree of affinity with interests will be easier with AsA. the user. At the same time, a user’s avatar could independently decide to visit places that may be particularly interesting to the user. Origins C) Users will always decide whether their avatar should follow their steps in the city or Sur de longues périodes de l'histoire, avec tout le be free to pursue their own interests (initiate mode d'existence des communautés humaines, contact with other avatars, visit other places, on voit également se transformer attend cultural events, etc.) Additionally, leur façon de percevoir. users will also be able to check the activity of La manière dont opère la perception - le médium dans lequel elle s'effectue - their avatar at any given time, and avatars ne dépend pas seulement de la nature humaine, themselves will also have the option of mais aussi de l'histoire.4 sending users a message whenever they encounter somebody. Finally, users will also Our recent history is not an exception to be able to view the trajectory of their avatar Benjamin's postulate. Indeed, perception has and compare it to their own. clearly changed along with the evolution of To sum up, users will have control over their time and space. Communication itself is also avatar at all times to check their interaction constantly shifting, and mental and physical with the environment, their contact with other representations continually morph into people and their participation in different different ways of expression. The human events. But they will also be able to simply desire to extend the body and the senses has observe the independent actions of their reached a point in which any kind of progress avatar in the simulated world. This way, users is conceived as possible by the collective will have the chance of comparing their imagination, even the most chimerical actions to those of their avatar. achievement. The digital worlds, their avatars, The avatar characterization will have two the social networks and their profiles, are only essential parameters: a few examples of the possibilities that new • a system of interest or affinity to filter technologies have opened up for us. They their decisions (whether to interact have created novel means of communication with somebody, visit a place, etc.). and innovative experiences, with new roles, • a personalization which will new representations for ourselves (or our alter determines the interaction of the egos), and new conceptions of the other avatar with other avatars (for members of our community. example, a user’s avatar will only This contribution is in the field of Information find the people the user already and Communication Sciences, and it consists knows, or they will be open to meet on the analysis of these changes, in order to new people who share the same interpret the causes and the effects of this interest, or they will talk with friends of a friend, etc.). 4 BENJAMIN, Walter, "L'œuvre d'art à l'époque de sa reproductibilité technique", Version de 1939, FolioPlus, 2007, page 16.

148 phenomena, to describe in detail their as a [real] person."6 On this level, we can advantages and disadvantages. But also, our estimate that sometimes, a digital task is to think and contribute towards representation with a rich history in a different uses and to find the best ways with particular virtual community could become an which people can obtain more benefits from entity in itself. As a player stated towards her technology. avatar: "Her character is, she says, [a very Theoretically, this work is partly devoted to beautiful creature and (I) would not want to the influence that digital self-representations shame her character since she has been in the (avatars) may have towards their creators (the company of my imagination for years.]"7 We users); to the relationship between society and could add to this that, in this case, it is the digital communities; to the feedback between materialization of the player's imagination: it the traditional and the non-traditional (hybrid) is indeed the image that is playing an worlds. After having analysed digital worlds important role. It clearly shows the case of an and their social, cultural and educational avatar functioning as a real extension of the effects on society, we propose to set up an self and that is why it has the potential to experimentation of a social device with the produce this kind of attachment. potential to generate a strong relationship A user or player will always have at least a between users and their avatars. minimal impression of being represented by AsA's main hypothesis believes that an avatar his or her avatar, and it is expected because can be considered as a dynamic extension of this is its purpose: to represent an entity, a the body; that it can bring original methods character. We can also suggest that an avatar for cognitive development and, last but not will probably leave traces that can tell us who least, that it can open up the way into a new his creator is. However, could we leave clues dimension for social interaction. Sherry to our avatar? Could we benefit from these Turkle affirms that "mirrors, literal and clues? Could an avatar expand our social and metaphorical, play an important role in human cultural background at any time and without a development. In literature, music, visual art, constant control from the user? These are the or computer programming, they allow us to questions that we are eager to answer with see ourselves from the outside, and to AsA. objectify aspects of ourselves we had perceived only from within."5 Indeed, our recent results have shown that an avatar is not What we expect from this experimentation? at all a mere digital representation of the self. An avatar, depending on its degree of "La virtualisation n'est plus une manière de evolution, can have a huge influence on the préférer la représentation psychique sur la user's physical life. réalité. In fact, thinking about a preconfigured avatar Bien au contraire, elle est un processus qui in terms of character and personality could be multiplie les représentations à partir d'un stimulus unique, linked to a user's own personal representation. permet de changer d'angle d'attaque For example, "in a role playing game, a dans la résolution d'un problème, player's central aim is the cognitive de se dégager de ses habitudes de pensée, development of their character. Although et finalement d'innover, à condition bien entendu statistics, nice weapons, and an interesting qu'elle soit suivi d'une actualisation look may aid in this, the real engagement 6 from most role players comes through the MaCCALLUM-STEWART, E. et PARSLER, J., The difficulties of Playing a Role in World of Warcraft, en formation and development of their character "Digital Culture, Play and Identity", The MIT Press, 2008, page 243. 5 TURKLE, Sherry, "The Second Self: Computers and 7 HAGSTROM, Charlotte, Gaming and Naming in the Human Spirit", The MIT Press, London, 2005, page World of Warcraft, en "Digital Culture, Play and 161. Identity", The MIT Press, 2008, page 272.

149 qui permet d'en recueillir les fruits."8 In conclusion, AsA can be an innovative It is important to highlight that we do not opportunity to widen our field of study in want to focus this research on the sentimental human sciences. Further progress may be reactions that can be simulated by an avatar. made as we continue to enrich our perception Instead, we want to observe human reactions of the world. from the autonomous actions of an avatar. These could lead human social practices to follow an alternative pattern, like "waiting for References my avatar to make a first contact" before • BENJAMIN, Walter, "L'œuvre d'art à meeting new people in the physical l'époque de sa reproductibilité technique", dimension. AsA could also bring to light an Version de 1939, FolioPlus, 2007. erroneous representation made by the user. • BORGES, Jorge Luis, The Garden of That would lead us to think about the Forking Paths in "Labyrinths", New accuracy of our own thoughts about Directions, 2007. • ourselves. CORNELIUSSEN, Hilde et WALKER This kind of experimentation could help RETTBERG, Jill, "Digital Culture, Play and Identity", The MIT Press, 2008. cognitive and social developments. In • addition, AsA could enhance digital TISSERON, Serge, "Rêver, fantasmer, virtualiser - Du virtuel psychique au communities in a way that is unprecedented: virtuel numérique", Dunod, 2012, overlying the physical space in real-time will (Edition Kindle). impulse this social tool towards another • TURKLE, Sherry, "The Second Self: communicational dimension. AsA may reach Computers and the Human Spirit", The a place that other social networks or digital MIT Press, London, 2005. communities have not yet achieved.

AsA's hypotheses and expected results are ambitious: • It may be use as a first contact encounter in physical life, leaving in some cases the decision for a future event to the avatar. • Social networks may benefit from this kind of experimentation, as it could make digital life more attached to physical spaces. • It may enrich user's activities in the physical world depending on the virtual path taken by his avatar. • It may establish and cultivate social interactions with friends and potential friends nearby the user's itinerary (through profiles and a system of affinity). • Academic poles may have alternative fields for socio-cultural research.

8 TISSERON, Serge, "Rêver, fantasmer, virtualiser - Du virtuel psychique au virtuel numérique", Dunod, 2012, location 69 - 87 de 3031 (Edition Kindle).

150 Digital Environment to Envision and Experience the Art of Light and Space

Cheng-Hsiu CHUANG Nan-Ching TAI Department of Architecture, Tamkang University Graduate Institute of Interactive Media Design, No.151, Yingzhuan Rd., Tamsui Dist. National Taipei University of Technology New Taipei City, Taiwan, 25137 No.1, Sec. 3, Chung-hsiao E. Rd., Taipei, Taiwan, 10608 [email protected] [email protected]

Abstract perceived depth for the audience. The Installations of Light and Space art often present characteristic of the “forcefulness” of an optical no objects but offer instead a perceptual illusion ensures that the illusory effect from a experience to the viewer. The light is directly particular arrangement can be expected for a used as material in a space to evoke a particular viewer with normal vision, and thus optical illusory visual perception; the work is thus illusions have been incorporated in many visual difficult to present in an alternative artist works [6]. The Ames Chair Demonstration representational media that is limited to is a recent example of Trompe l’oiel that utilizes displaying the physical reality of light and the a geometrical illusion to create a peepshow perceptual reality of how it is perceived by the installation in which a collection of random human visual system. Thus, trial and error is the objects can appear to be a chair from a primary approach to creating Light and Space art, and the art can only be experienced by particular viewpoint [4]. The Art of Light and visiting the actual installation site. This study Space, a major development in modern art proposes a computational framework that starting in the 1960s, on the other hand, integrates physically based rendering, employs optical illusions that arise from more perceptually based tone-mapping, and stereo complex visual perceptions [1]. imaging techniques to generate a digital The cause-and-effect relationship of the representation that can provide a pictorial geometrical illusion employed in the Ames Chair environment to envision and experience the art Demonstration has been well established, and a of Light and Space. representation of such an illusion does not require high visual realism. Similar design Keywords works can thus be depicted in representational Optical Illusion, Art of Light and Space, Lighting media to test their capacity for conveying Simulation, High Dynamic Range Imagery, illusions. However, the Light and Space artists Stereo Image. utilize perceptual effects that are far more complex than the one used in the Ames Chair 1 Introduction Demonstration, such as the retinal afterimage in Retinal Replay Volume by Robert Irwin and the An optical illusion refers to an error of the color compensation in Arhirit by James Turrell visual experience that occurs systematically [1]. between the visual stimulus and perceptual These optical illusory effects are difficult to judgment [6][8]. It can be generalized as the represent on planar media. As a result, an discrepancy between physical reality and the empirical method of trial and error is the visual perception of such physical reality [3]. primary approach to creating this type of Optical illusions have long been known, and modern art, and the art can only be experienced their various applications can be observed in art by visiting the actual installation site. and architecture. The two-dimensional patterns In this study, we examine the possibility of of the mosaic floor from the Antioch are an early representing the optical illusory effect used in example of creating illusory three-dimensional James Turrell’s Afrum, in which light forms an forms on a planar surface; the spatial illusory solid form. The intention is to develop a configuration in Theatro Olimpico, on the other computational framework that can generate a hand, utilizes false perspective to amplify the pictorial representation that reflects the optical

151 reality of visual illusions, to mediate the design process of installations of Light and Space art, and to extend its perceptual experience from the physical site to the virtual environment.

2 Methodology The design process is often mediated on representational media where the perceptual experience can be envisioned from the conceptual configuration. Figure 1 illustrates the design process of an art installation attempting to recreate an exhibition similar to the Ames Chair Demonstration. As the geometrical relation from the conceptual views Figure 2: Google SketchUp simulation of the can be fairly accurately previewed in the artwork similar to James Turrell’s Afrum perceptual view in simple modeling software such as Google SketchUp, such a representation provides the necessary visual realism to As a visual illusion is based on the mediate the design process of this type of art discrepancy between physical reality and its installation. However, since Light and Space art perception, to faithfully represent a visual utilizes a complex visual process, the illusion, it is essential that the representation representation offered by Google SketchUp, as encompasses the physical reality of the illustrated in figure 2, falls short of exhibiting represented scene and be capable of stimulating the intended optical effect of the artwork. the perceptual response just as the real scene does. Physically based rendering has been developed to allow the pixels of the image to store the actual lighting data of the scene [9] [13]. Several rendering software such as RADIANCE, LuxRender, and Maxwell Render, to name a few, are physically based simulation and visualization software that aim to simulate the complex lighting interaction of the real world in the virtual environment. Among them, RADIANCE has been validated by scientific investigation [7][11] and is thus chosen in this study to generate the scene to visualize the optical effects. RADIANCE outputs its final rendering in HDR file format, which can contain a luminance range that covers the complete range that humans can perceive, from 10-6 to 108 cd/m2. However, the HDR image cannot be displayed on a conventional display device, which is limited to a luminance range of two to three orders of magnitude. Tone-mapping is a computational method developed to compress the dynamic range of HDR images to a low- dynamic-range image [12]. Many tone-mapping Figure 1: Conceptual configuration and operators (TMO) were developed, and the perceptual experience of an art installation photographic TMO [10] is chosen as it has designed to create an exhibition similar to consistently provided good results in various Adelbert Ames Jr.’s chair demonstration perceptual studies [2][5][15].

152 In general, computer rendering is generated offset evenly from the viewpoint location, from a single-camera viewpoint, which fails to pointing toward the same focal point as in the address the retinal disparity of our visual single-camera setting. system. Stereopsis resulting from retinal disparity is a major factor that contributes to the geometrical illusion of three-dimensional form. Several computational techniques have been developed to create stereo images on digital displays. The common strategy is to create slightly different images for the left eye and the right eye, and to present the images to the corresponding eyes simultaneously to create stereo visual perception. Anaglyph technology was used in this study based on its affordability and the simplicity of the imaging process. The red and cyan color channels in the image can be eliminated accordingly for the images for the left and right eyes using an image editing program, and the modified images can be presented to the viewer’s left and right eyes, which are filtered by anaglyph glasses (which Figure 3: Configuration of the installation apply a red filter to the left eye and a cyan filter to the right eye). As a result, a pair of physically accurate HDR scenes can be tone-mapped to displayable low-dynamic-range images and presented to the viewer stereoscopically on a common display device.

3 Simulations Figure 3 illustrates the configuration of an art installation design that resembles James Turrell’s Afrum. A 360 × 360 × 240 cm room is enclosed from all sides except for an irregularly shaped opening located at a height of 55 cm at the intersection of the two walls at the north end. Double walls were configured for the pair of walls that intersect at the opening. The double walls are spaced 60 cm apart, forming a chamber that admits daylight from two open ends, allowing the reflected natural light to come through the opening at the intersection Figure 4: Detailed configuration of the void and into the interior space. The wall thicknesses opening are 15 cm for the exterior wall and 30 cm for the Figures 5 and 6 illustrate the final renderings interior wall. The shape of the void opening is of both the single- and the two-camera settings based on how it would appear from the for monocular and binocular vision, perspective of the pre-determined viewpoint at respectively. The scenes were rendered by normal eye level when sitting on the chair RADIANCE and tone-mapped by the shown in the figure. The viewpoint is set at a photographic tone-mapping operator. Figure 5 height of 110 cm and a distance of 300 cm from shows the image generated by the single- the opening, with the viewer looking at the camera viewpoint; figure 6 shows the stereo center of the opening. To capture the stereo image assembled from the two images image, two cameras were spaced 6.5 cm apart, generated by the two-camera setting.

153 (a) (b)

Figure 5: Rendering from a single-camera setting (c) (d) Figure 7: a) Monocular view generated by Google SketchUp. b) Binocular view generated by Google SketchUp. c) Monocular view generated by the tone-mapped image rendered by RADIANCE. d) Binocular view assembled by tone-mapped images rendered by RADIANCE

Google SketchUp offers a realistic visual representation of the geometrical relations, presenting fairly accurate lines and edges of the formal configuration, but it reveals no visual information about the lighting distribution. Neither the monocular nor the binocular representation can demonstrate the optical effect of the art installation. On the other hand, Figure 6: Rendering assembled from images the image set generated by the physically based generated by the two-camera setting rendering and perceptually based tone-mapping illustrates the optical effect, with the monocular view suggesting the illusory three-dimensional 4 Discussion form; the effect was more convincing when the Figure 7 shows comparisons of the perceptual binocular rendering was viewed with the experience of the art installation from the anaglyph glasses. It is thus concluded that the expected viewpoint generated by different computational framework that integrates computer simulation tools. Figures 7a and 7b physically based rendering, perceptually based illustrate the monocular and binocular views tone-mapping, and stereo imaging techniques respectively generated by Google SketchUp, provides a promising approach for generating a while 7c and 7d illustrate the same set of views digital environment that can mediate the design simulated by the computational framework that process of an art installation of Light and Space. integrates the RADIANCE and photographic The Afrum by James Turrell uses a stable tone-mapping process. electric light source to sculpt out the three- dimensional illusory form. With the availability

154 of the digital environment to preview the 5 Conclusion conceptual configuration and the resulting Every representational system is developed to perceptual experience, more complex offer different interactions and visual realism interaction can be incorporated into the design for a particular function [14]. In this study, we considerations. The art installation presented in have explored a computational framework that this paper attempts to utilize the ever-changing can generate a pictorial environment to help daylighting as the light source to create a more visualize and experience the design process and dynamic artistic experience. Figure 8 illustrates result of an installation of the art of Light and the sequential renderings of a modified design, Space. The art installations of Light and Space with the two void open ends covered by often present no objects but instead offer a additional green and purple glass, rendered perceptual experience to the visitor. The light is under the same CIE sky condition at different directly used as the material in a space to evoke times (8:00 a.m., 10:00 a.m., 12:00 p.m., 2:00 a particular illusory visual perception. In this p.m., 4:00 p.m., and 6:00 p.m.) to simulate how study, we propose a computational framework the art installation interacts with the daylight that integrates physically based rendering, over time. As shown in figure 8, the simulations perceptually based tone-mapping, and stereo demonstrate that the illusory solid form imaging techniques to generate a digital changes color throughout the day, suggesting environment in which both the physical and that the art installation can be presented at the perceptual reality of how light is distributed in a same location in different ways depending on space and how it is perceived by the human the daylighting. visual system can be simulated. It is important to note that we value the experimental method of creating the art of Light and Space, and this work is by no means seeking an alternative approach to replace the existing one. The work presented here is a beginning to explore the possibility of how the art of Light and Space can be advanced in the Post-Digital Art era. The immediate future work is to refine the computational framework for simulating the art 8:00 a.m. 10:00 a.m. of Light and Space by validating the simulation with an actual physical installation. This work has demonstrated the possibility that the proper computational capabilities can facilitate a certain amount of trial and error in the design process, and allow more complex lighting interaction to be incorporated into the design considerations of the art of Light and Space. Also, the same environment can more 12:00 p.m. 2:00 p.m. realistically depict the visual aspects of the resulting work, extending the viewing experience from the actual physical site to the virtual world. Figure 9 illustrates a possible application to sculpting illusory spatial geometries, formed by light and reflected light and utilizing the optical reflection from a physical mirror installation. With the availability of a digital environment 4:00 p.m. 6:00 p.m. that can reflect the visual realism of the perception of light and its visual illusions, the Figure 8: Art installation interacting with conceptual design can be first experimented daylight throughout a single day with and refined in the digital environment, and

155 the final work can be also presented in the very [5] Jiangtao Kuang, Hiroshi Yamaguchi, same virtual realm in addition to the real site Changmeng Liu, Garrett M. Johnson, and Mark exhibition. D. Fairchild. 2007. “Evaluating HDR Rendering Algorithms.” ACM Transactions on Applied Perception, 4, no. 2, article no. 9. [6] Matthew Luckiesh. 1965. Visual Illusions: Their Causes, Characteristics and Applications. Dover Edition. Dover Publications, New York. [7] J. Mardaljevic. 2001. “The BRE-LDMP Dataset: A New Benchmark for the Validation of Illuminance Prediction Techniques.” Lighting Research and Technology, 33, no. 2, pages 117–136. [8] Stephen E. Palmer. 1999. Vision Science: Photons to Phenomenology. 1st ed. The MIT Press, Cambridge, Massachusetts. [9] Matt Pharr and Greg Humphreys. 2004. Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann, San Francisco. [10] Erik Reinhard, Michael Stark, Peter Shirley, and James Ferwerda. 2002. “Photographic Tone Reproduction for Digital Images.” in ACM Transactions on Graphics, pages 267–276. San Antonio, Texas. [11] Alexa Ruppertsberg and Marina Bloj. 2006. “Rendering Complex Scenes for Psychophysics Using Radiance: How Accurate Can You Get?” Journal of the Optical Society of America, 23, no. 4, pages 759-768. [12] Erik Reinhard, Greg Ward, Sumanta Figure 9: Design sketch and simulation of an Pattanaik, and Paul Debevec. 2006. High art installation interacting with light and the Dynamic Range Imaging: Acquisition, Display, reflected light and Image-Based Lighting. Morgan Kaufmann, San Francisco. [13] Greg Ward and Rob Shakespeare. 1998. References Rendering with Radiance: The Art and Science [1] Jan Butterfield. 1993. The Art of Light and of Lighting Visualization. Booksurge, LLC, Space. Abbeville Press, New York. Davis, CA. [2] Martin Cadík, Michael Wimmer, Laszlo [14] John Willats. 1997. Art and Representation: Neumann, and Alessandro Artusi. 2008. New Principles in the Analysis of Pictures. “Evaluation of HDR Tone Mapping Methods Princeton University Press, Princeton, NJ. Using Essential Perceptual Attributes.” in [15] Akiko Yoshida, Volker Blanz, Karol Computer Graphics, 32, no. 3, pages 330–349. Myszkowski, and Hans-Peter Seidel. 2005. [3] Ronald G. Carraher and Jacqueline B. “Perceptual Evaluation of Tone Mapping Thurston. 1966. Optical Illusions and the Operators with Real-World Scenes.” in Human Visual Arts. Van Nostrand Reinhold, New York. Vision & Electronic Imaging X, SPIE, pages 192–203. San Jose, CA. [4] E. H. Gombrich. 2000. Art and Illusion. Princeton University Press, Princeton, New Jersey.

156 Art images classiﬁcation using Bag-of-Visualterms representation

Reverter, F; Rosado, P; Figueras, E and Planas, M.A. University of Barcelona Diagonal 643, 08028 Barcelona, Spain, freverter,eﬁgueras @ub.edu, prrforma,miquelplanas @gmail.com { } { }

Abstract work. Additionally, we explore the capability of We explore the problem of classifying scene cate- this kind of modelization to become useful for gories that are the basis of the ideation and design- the production of software art. ing the sculptural production of an artist. The main goal is evaluate the performance of SIFT descrip- 2 Image Representation and tors, the Bag-of-visualterms representation and the Matching spatial pyramid matching when these computer vision methodologies are faced with this type of im- The BOV representation was first used [Willam- ages. Results are promising, on average the perfor- owski, J;Arregui, D; Csurka, G; Dance, C and mance score is about 70% and its standard deviation Fan, L., 2004] as an image representation for is about 5%. an object recognition system. In the BOV representation, local descriptors fj are quan- Keywords tized into their respective visterms vi = Q(fj) Bag-of-visterms, SIFT descriptors, pyramid match and used to represent the images from which kernel. they were extracted. The quantization process groups similar descriptors together, with 1 Introduction the aim that the descriptors in each resulting Image representation is a very important ele- group arise from local patterns with similar vi- ment for image classification, annotation, seg- sual appearance. The number of occurrences mentation or retrieval. Nearly all the methods of each group/visterm in a given image is the in computer vision which deal with image con- elementary feature of the BOV representation. tent representation resort to features capable of More precisely, the BOV representation is the representing image content in a compact way. histogram of the various visterms’ occurrences. Local features based representation can produce To construct the BOV feature vector h from a versatile and robust image representation ca- an image I four steps are required. In brief, lo- pable of representing global and local content cal interest points are automatically detected in at the same time. Describing an object or scene the image, then local descriptors are computed using local features computed at interest loca- over the regions defined around those local in- tions makes the description robust to partial oc- terest points (occasionally local descriptors may clusion and image transformation. This results be computed on a dense grid over the image in- from the local character of the features and their stead over local interest points). After this ex- invariance to image transformations. traction step, the descriptors are quantized into The bag-of-visterms (BOV) is an image rep- visterms, and all occurrences of each visterm of resentation built from automatically extracted the vocabulary are counted to build the BOV and quantized local descriptors refereed to as representation of the image. visterms in the remainder of this paper. The BOV representation, which is derived from 2.1 Feature extraction these local features, has been shown to be one of The BOV construction requires two main de- the best image representations in several tasks. sign decisions: the choice of local descriptors The main objective of this study is assessing that we apply on our images to extract local the performance of SIFT descriptors, Bag-of- features, and the choice of which method we use visualterms representation and spatial pyramid to obtain the visterms’ vocabulary. Both these matching for automatic analysis of images that choices can influence the resulting system’s per- are the basis of the ideation and designing of art formance. Nevertheless BOV is a robust image

157 representation, which retains its good perfor- This vector-space representation of an image mance over a large range of parameter choices. contains no information about spatial relation- For better discriminative power, we utilize ships between visterms, in the same way the higher dimensional features which are SIFT standard bag-of-words text representation re- (Scale Invariant Feature Transform) descriptors moves the word ordering information. introduced by [Lowe, 2004]. The SIFT descrip- For such whole-image categorization tasks, tor is a histogram based representation of the bag-of-features methods, which represent an im- gradient orientations of the gray-scale image age as an orderless collection of local features, patch. Then, our features are SIFT descriptors have recently demonstrated impressive levels of of 16 16 pixel patches computed over a regular performance. However, because these methods × grid with spacing of 8 pixels. disregard all information about the spatial lay- Our decision to use a dense regular grid in- out of the features, they have severely limited stead of interest points was based on the com- descriptive ability. In particular, they are in- parative evaluation of [Fei-Fei, L, Perona, P, capable of capturing shape or of segmenting an 2005], who have shown that dense features work object from its background. better for scene classification. Intuitively, a dense image description is necessary to capture 2.3 Spatial matching scheme uniform regions such as sky, calm water, or road To overcome the limitations of the bag-of- surface. SIFT was also found to work best for visterms approach we have implemented the the task of object classification [Sivic, J., Rus- pyramid histogram of visual words (PHOW) de- sell, B. C., Efros, A. A., Zisserman, A., and scriptor introduced in [Bosch, A., Zisserman, Freeman, W. T, 2005] and [Quelhas, P., Monay, A., Munoz, X. , 2007], which is based on spatial F., Odobez, J.-M., Gatica-Perez, D., Tuyte- pyramid matching [Lazebnik, S., Schmid, C., laars, T., and Gool, L. V., 2005]. and Ponce, J, 2006] and [Grauman and Darrel, 2.2 Visual Vocabulary 2005]. Informally, pyramid matching works by placing a sequence of increasingly coarser grids In order to obtain a text-like representation, we over the feature space and taking a weighted quantize each local descriptor s into one of a dis- sum of the number of matches that occur at crete set of visterms v according to a nearest V each level of resolution. At any fixed resolu- neighbor rule: tion, two points are said to match if they fall into the same cell of the grid; matches found at s Q(s) = v dist(s, v ) dist(s, v ), 7→ i ↔ i ≤ j finer resolutions are weighted more highly than matches found at coarser resolutions. for all j = 1, ..., N , where N denotes the size of the visterm set.V V The similarity between a pair of images I and We will call vocabulary the set of all the J is computed using a kernel function between visterms. The vocabulary constructionV is per- their PHOW histogram descriptors DI and DJ , formed through clustering. More specifically, with appropriate weightings for each level of the we apply the k-means algorithm to a set of lo- pyramid: cal descriptors extracted from training images, 1 and keep the means as visterms. We used the K(D ,D ) = exp α d (D ,D ) I J {β l l I J } Euclidean distance in the clustering and quanti- l L X∈ zation processes, and choose the number of clus- ters depending on the desired vocabulary size. where β is the average of l L αldl(DI ,DJ ) ∈ Finally, the bag-of-visterms (BOV) represen- over the training data, αl is the weight at level l 2 P tation is constructed from local descriptors ac- and dl is the χ distance [Zhang,J., Marszaek,M. cording to: , Lazebnik, C., and Schmid, S, 2007] between DI and DJ at pyramid level l computed using h(d) = (n(d, v1), n(d, v2), ..., n(d, vN )) on the normalized histograms at that level. V The spatial histograms could be used as im- with n(d, vi), i = 1, ..., N , denotes the number age descriptors and fed to a linear SVM clas- V of occurrences of visterm vi in image d. To classifier. Linear SVMs are very fast to train, but sify an input image d represented either by the also limited to use an inner product to com- bag-of-visterms vector h(d) we employed Sup- pare descriptors. Much better results can be port Vector Machines (SVMs). obtained by computing an explicit feature map

158 that ”emulates” a non linear χ2-kernel as a linear one [Vedaldi, A., and Zisserman, A., 2010]. 3 Results In this paper we propose to automatically analyze images from a database of photographs by Dr. M.A. Planas Rossell (Pro- fessor of sculpture, University of Barcelona). The database consist of 150 images previously classified in 5 categories: Central architecture (CA),Geometric stone(GS),Irregular stone(IS),Textured stone(TS) and Silhouettes (SI). These categories correspond to 5 differ- Figure 1: A sample from the dataset of images. ent typologies identified in the photographic images from the database. Images are the basis of of classifying images into categories very effi- the ideation and designing the sculptural work ciently. In this work we have explored the be- of the artist. Figure 1 shows a sample of the havior of some of these techniques when faced database in our experiment. We have analyzed with a database of images whose category are a training dataset of 75 images, 15 images from not determined by the objects they contain but each category. From this dataset we have build rather respond to conceptual categories in the a vocabulary of 300 visterms. Then we have ideation process of the work of an artist. We computed the PHOW descriptors of each image. conclude that although the database analyzed Finally, we compute the feature map associated in our study is quite specific, results in the clas- with the χ2-kernel and estimate the multiclass sification are very promising and comparable to SVM classifier. Efficient code to compute our the results of other studies reported in the lit- feature maps is available as part of the open erature dealing with database of images with source VLFeat library [Vedaldi, A., and Fulker- object categories. son, B., 2008]. We classify a set of test images (75 images; 15 5 Acknowledgements images from each category). The classification process is repeated 10 times (changing the train- This work was partially funded by the Univer- ing and test sets), and the average performance sity of Barcelona grant APPCSHUM 2011-2012. score and its standard deviation are 70.67% and References 5.03%. Table 1 shows a sample of the misclassi- fication rates obtained in the analysis of the 10 Bosch, A., Zisserman, A., Munoz, X. . 2007. test datasets. Image classification using random forests and ferns. In Proceedings of IEEE International CA GS IS SI TS Conference on Computer Vision (ICCV). CA 12 0 1 2 0 Fei-Fei, L, Perona, P. 2005. A bayesian hier- GS 1 7 5 1 1 archical model for learning natural scene cat- IS 0 4 9 0 2 egories. In Proceedings of CVPR. SI 2 0 0 13 0 TS 0 0 6 0 9 K. Grauman and T. Darrel. 2005. The pyramid match kernel: Discriminative classifica- Table 1: True category in rows and Predicted tion with sets of image features. In Pro- category in columns. ceedings of IEEE International Conference on Computer Vision (ICCV), Beijing. Lazebnik, S., Schmid, C., and Ponce, J. 2006. 4 Conclusions Beyond bags of features: Spatial pyramid matching for recognizing natural scene cat- The problem of classifying images by the object egories. In Proceedings of CVPR. categories they contain constitutes an area of great activity in computer vision research. The D. G. Lowe. 2004. Distinctive image features result of this research is that currently available from scale-invariant keypoints. International a set of methodologies that address the problem Journal of Computer Vision, 60(2):91110.

159 Quelhas, P., Monay, F., Odobez, J.-M., Gatica-Perez, D., Tuytelaars, T., and Gool, L. V. 2005. Modeling scenes with local descriptors and latent aspects. In Proceedings of IEEE International Conference on Com- puter Vision (ICCV), Beijing. Sivic, J., Russell, B. C., Efros, A. A., Zisser- man, A., and Freeman, W. T. 2005. Discover- ing objects and their location in image collections. In Proceedings of IEEE International Conference on Computer Vision, Beijing. Vedaldi, A., and Fulkerson, B. 2008. Vlfeat library. http://www.vlfeat.org/. Vedaldi, A., and Zisserman, A. 2010. Eﬃcient additive kernels via explicit feature maps. In Proceedings of CVPR. Willamowski, J;Arregui, D; Csurka, G; Dance, C and Fan, L. 2004. Categorizing nine visual classes using local appearance descrip- torsl. In Proceedings of LAVS Workshop, in ICPR04, Cambridge. Zhang,J., Marszaek,M. , Lazebnik, C., and Schmid, S. 2007. Local features and kernels for classiﬁcation of texture and object categories: a comprehensive study. IJCV.

160 Post Anxiety Art: Economies and cultures of digital painting

Jo Briggs Mark Blythe School of Design, Northumbria University School of Design, Northumbria University Newcastle upon Tyne, UK Newcastle upon Tyne, UK [email protected] [email protected]

Abstract Hockney’s team use very large files to create prints Digital technologies such as the iPad, apps and for exhibition and the market (ibid). content sharing websites present novel ways of producing and disseminating art. Searches for iPad However, control of digital files is problematic, as art on Flickr revealed a vast archive with one the ongoing crisis in the music industry "Brushes" app group producing more than 41,000 demonstrates. There are then economic questions images. Content analysis suggests that the work for artists using digital media, but there are also conformed for the most part to standard art questions of cultural legitimacy. taxonomies: portraits, figurative, flower, etc. This paper explores questions of the economic value and As part of an ongoing project investigating the cultural anxiety around new and emerging notion of a digital original (Digital Originals practices. website) a workshop was held where seven artists sketched from a still life using iPads. One of the Keywords artists, Hilary, reacted with a strong exclamation of Digital painting, iPad, social media, identity, digital frustration. The device was "a way to make... economy. diaorreah" she cried, with apps determining that everyone's art has "the same marks, same look". Introduction Hilary wanted something novel, perhaps something Apple released the first iPad in April 2010, selling more authentic: "I want to see some-thing come out three hundred thousand units on day one and three of this machine that I've never seen before." Hilary million in eighty days (Wikipedia iPad). When utilises a range of materials in her urban shared installed with one of the many available painting studio space and admits that she has yet to fully apps, such as Procreate or Brushes, the device engage with digital tools. She was quick to offers the aspiring or established artist a combined disassociate herself from what she perceived to be tool and medium through which to conveniently "low-brow" cultural phenomenon, in the process generate art. The British artist David Hockney has seemingly revealing resolutely "highbrow" tastes received much attention for his iPad and iPhone (Bourdieu 1984, 1990). drawings and paintings, not least because they present a challenge to traditional notions of what an original work, or limited edition print is. Recent journalism captures this anxiety:

A friend of mine, the historian Michael Burleigh, boasted that he had acquired a collection of 65 Hockneys — iPad drawings can easily be emailed, and Hockney had taken to sending them out to his friends [...] I have a collection of 20. I Figure 1. Hilary's first iPad painting can [...] send them to my friends. The This paper explores the questions of both cultural country is now littered with original and economic value by considering established, Hockneys. This raises an awkward emerging and aspiring artists' uses of the device and question. “I don’t know how we’re going to the new venues that have emerged for displaying make any money out of this", he [Hockney] and archiving digital paintings. The paper presents mutters, the Yorkshire intonation flattening a content analysis of work produced on the iPad even further. (Appleyard 2011) and shared on the image sharing site Flickr, beginning with a small sample of returns on a Appleyard claims that "in fact, he does", because search for the term "iPad art". This reveals a wide the emailed files are very low resolution, though range of content which is further explored in more

161 focused searches within a Brushes app users' group demonstrate interesting configurations and uses of on Flickr. These returns are further categorised and exploited tools. explored in depth, with examination of a sub-set of However, increasingly software tools and self-portraiture informed by cultural and social generative devices are "locked down" facilitating theory. use by a novice but perhaps obstructing access to

fine technical and aesthetic control. Finger painting Apps such as Procreate and Brushes are Established artists programmed to simulate painted and drawn marks The iPad and Brushes app have generated artifacts made with the finger or a stylus on the touch that have received critical acclaim. Hockney's sensitive screen. Marks made mimic some of the practice now incorporates what he calls iPad physical properties of material paint such as drawings that are exhibited internationally (Royal fluidity. The palette is only limited by the Academy and Guggenheim Bilbao websites). significant display affordances of the iPad screen, YouTube reveals that another early adopter of the enabling the digital painter to access a spectrum of Brushes app to capitalise is animator Shawn Harris. pigments unlimited by expense or conservational He combined hundreds of individual iPad paintings instability. These paintings are never sullied by in the production of a music video animation for undisciplined use of an artists’ mixing palette and British girl group, Blush (Blush on Youtube). The colours remain vibrant however many layers are video generated a level of positive mainstream added. media coverage not usually associated with urban black music (e.g. Madison 2011). Autodidact digital artists Educationalist David Buckingham has noted that Otherwise, YouTube is awash with what might be software interface design typically facilitates called didactic time-lapse videos of iPad painting creative production whereby the "process of and drawing techniques. These utilise a function constructing meaning (is) clearly visual" (2007: within some apps that captures durational image 167) and the activity of selecting, editing and making, mark by mark. Occasionally, the URL of manipulating images is one of "visual thinking" an e-commerce site is shown, advertising for sale (ibid 168). Production is intuitive, iterative and printed copies of the final painting. Preliminary provisional (most apps have a multiple "undo" web searches suggested that while the time-lapse function) and this has contributed to a collapse of painting videos are posted to YouTube, a digital file the traditional production cycle. The creative of the finished still images is posted to sites such as producer can "get stuck in" and begin generating Flickr, and this became the focus of the enquiry. artifacts with little preliminary planning — or Content analysis of the top 100 returns within training — pushing most of the activity along the Flickr for "iPad art" production cycle towards postproduction design At the time of writing the search “iPad Art” (see Burn and Parker 2003; Kress and van Leeuwen returned more than eleven and a half thousand 2001). However, Software tools' particular effects results. These were sorted by the “most interesting” filters etc., especially if left at "default" setting, Flickr filter and the first one hundred were sampled may dominate inexperienced producers’ artifacts on 25th May 2012. Although this is a very small (Briggs 2010: 91–92). sample scanning the subsequent pages indicated Research in the field of Human Computer that the first hundred interesting returns were not Interaction (Bardzell 2007) suggests that digital entirely atypical. Figure 2 shows a page of results software tools have inherent "enabling factors" selected later in the data set. (ibid 13) leading them to be enthusiastically taken A Qualitative Content analysis (Mayring 2004) was up by amateur producers. In turn, amateurs’ use has performed. This method is frequently used in facilitated ‘tremendous artistic innovation’ (ibid) studies of mass media such as newspapers and such as machinima where video game environments advertisements (ibid). Items are coded and counted are used as stages for cinematic productions. to indicate patterns and trends in data sets (e.g. Bardzell says that amateurs' multimedia artifacts sexist imagery in advertising). In this analysis each must display aesthetic "elaboration" to be judged as image was categorised by artist, subject matter and being of "quality" (ibid 28). This suggests that work technique. "Most interesting" pictures within Flickr must have more than aesthetic merit per se and

162

Figure 2. Screen shot of page 22 of results from Flickr search “iPad art” are determined by an undisclosed algorithm that is lives and landscapes (Appleyard 2011). The art said to take into account: the number of views critic Martin Gayford (2007) sees in Hockney's (internal/ external), the number of comments posted iPhone drawings views on the self expanded by the and status of commenter, the number of times the ready availability and portability of the iPhone: image is tagged as someone's favourite (a form of What other painter has had occasion to intra-site bookmarking), the number of discussion record, as Hockney has, his naked foot groups a particular image is tagged to belong to beside a slipper when he got out of bed? Or (with more selective group association being his Bathroom taps with the window behind, favoured) and issues of "time varying behaviour" of a morning bather’s of the world, or the these factors (see Wesley Hein blog). washing-up in the sink of the Bridlington (Self) portraiture kitchen, a glass ashtray full of cigarette It is striking and unsurprising that portraiture, butts? (p195). including self-portraits (25 percent) and figurative The iPhone is, in some respects, more “ready to work (representing some of the human body but not hand” even than a notebook and paper. It is kept on a specific subject, 16 percent) constitute together 41 bedside tables and Hockney uses it to record percent of the "most interesting" images returned. sunrises in light conditions that would be too dark The next largest categories were: flower (9 for traditional media. The foot by the slipper and percent); animal (8 percent); landscape (8 percent). the view through darkened curtains of the dawn’s The other categories at 5 percent or less were: word first glimmering light suggest new kinds of subject. as image, abstract, still life, vehicle, caricature / cartoon, fantasy, figures in landscape and montage. However, the returns in this Flickr search are for the most part traditional head and shoulder Artists have traditionally pictured themselves in compositions with a significant number their work, partly due to convenience of being their functionally captured, as though for an online own subject/object (Doy 2005: 46). Another readily profile image or passport photograph. A majority of available visual source evident in the images the self-portrait paintings are reproduced from returned is "celebrity" media culture. A small sub- photographs. Of 376 images in the Brushes gallery, section of six portraits featured celebrities including tracing evidently generated 230. Typically, the Grace Kelly, Audrey Hepburn, Beyonce, photo-derived portraits seem to have been captured Humphrey Bogart, Marilyn Monroe and Sharbat by their maker working alone in a domestic setting. Gula, a young Afghan subject of a National Some subjects exhibit a seemingly distracted Geographic cover. "YouTube gaze" (Adami 2008) — apparently It's also worth noting that one of David Hockney's watching their image as it is recorded, possibly by a favourite iPad painting themes is self-portraiture, capture device (e.g. webcam) positioned askew with his two other dominant subjects being still from the screen. Other artists use an iPhone or iPad

163 close to the body or at extended arms length. Some categories (for example, relating to medium, genre, images reveal how the device helps generate ideas, subject matter, app used etc.). Sixty-eight focused for example appearing as a prop. One sketch shows searches were made on the 1st June 2012 to try to the shape of the fingers holding a mobile phone ascertain particular areas of interest among the (Figure 3). Brushes Gallery group, as evidenced by the tags used in describing work. The top ten identified classifications are presented in Table 1.

art 8583 drawing32% 44195% me29% 4385 45%digital 4137 illustration 3330 photo 2498 portrait 2292 abstract 1633 Figure 3 shows an image by Julia Kay artist 1311 The Brushes App and Flickr Brushes Gallery landscape 1051 The initial search on iPad art indicated that there was a wide variety of work archived on the site and Table 1: Returns within the Brushes community for suggested more focused searches around existing genre classification Flickr groups and communities. The same search For comparison, relevant search results relating to for iPad art revealed the existence of 1,954 art "movements", mediums and painting genre are discussion/ community groups. The largest given below in Table 2. focusing on art production is the Brushes Gallery – iPhone/iPad Art group, or Brushes Gallery, with Modernism 126 ink 182 3,918 members and 41,070 images. As well as 32%Pop 6237% oils 169photo being the largest group, the specific focus on the Impressionism 45 montage34% 42 Brushes app suggested this group's relevance for figurative 20 self portrait 376 enquiry into digital painting as opposed to more watercolour 233 still life 164 general iPad art practice. Brushes is a painting app designed specifically for the Apple devices (iPhone, Table 2: Returns within the Brushes community for iPod touch and iPad). Functions include: art history terms "Advanced color picker, several realistic brushes, Descriptors selected by members are general or multiple layers, extreme zooming, and a simple yet popular rather than appropriated from conventional deep interface" (Brushes website). The app also art history and practice discourse. Informed by the records all actions during the creative production earlier emergence of portraiture as a dominant process and stores these in a .brushes data file. This theme, the sample of 376 "self portraits" was can be posted elsewhere in video format selected for further analysis. Images were sorted representing the durational unfolding of the digital according to artist, subject matter and artistic style. painting (ibid). The rules of the Flickr Brushes These were examined, with particular focus on the Gallery membership are explicit. Posted artifacts most prolific self-portraitists. must be created using the Brushes app in combination with one of the Apple devices. Other Brushes' most prolific self-portraitists apps utilised must be clearly indicated, as should This small cohort of Brushes Gallery artists has images generated from photographs, though this each developed sequences of self-portraits, some rule is not always adhered to. extending back to September 2008. A representative image for the nine most prolific Analysis of the Brushes Gallery artists is shown in Figure 4, alongside Flickr user Flickr offers limited meta-tagging functionality name and number of self-portrait images produced determining that contributors must classify each in brackets. artifact, rather than selecting from pre-existing

164

Figure 4, Self portraitists on Brushes (no. images) from left Julia L. Kay (109), patricio villarroel (22), MiaBia_DC (18), frnjpn (13), ipad junkie (9), Taboogallery (9), bavaroland (8), hgberk (6), BLUEJAYWAY2011 (6). Amelia Jones in her 2006 book "Self/Image: whereby learning and sharing take place amongst Technology, representation, and the contemporary groups formed online (see Gee 2004). Members subject" describes how a "confluence of beliefs post comments about each other's work in a and discursive articulations" during the process of exchange and mutual influence forming Renaissance, led to the increased prominence of a shared collective identity. self-portraiture (ibid 5). She goes on to outline Sharing and Commenting how industrialisation and the development of The following section briefly considers the artist imaging technologies (including photography) statements and comments posted on the Brushes provided the means to reproduce art (ibid). This Gallery around the images in Figure 5. The self was important as it placed a "premium" not only portraits typically garner between ten and twenty on art but on the "source of creative genius", that comments which include dialogue with the artist is, individual artists (ibid). Similarly, Richard himself. The first image (14) in Figure 5 featuring Howells notes how modernism led to an the artist in a Batman Cowl is called “I Dream of understanding that all art might be considered as a Being Super”. BLUEJAYWAY2011 tags the form of self-portraiture, as the artifact reveals as image with an informal statement which includes much about its maker as the subject he/she reflective self criticism: chooses to represent (2003: 49). During this period, art became a vehicle for communicating I have just realised that it's my version of "deep emotions" to its publics, fostering a "special an Alex Ross Batman cover. Alex Ross is tie" back to the artist (ibid). the guy who does those huge superheroes — Superman on steroids! Hey with post Gen Doy, asserts that the physical act of painting modernism everything is ok! I'm still not helps construct and reaffirm ones’ artist identity entirely happy with my colours, and Sue is (ibid 59). She discusses how self-portraiture is right they worked well, with extreme highlights utilised by artists, sometimes over prolonged etc. it's all part of the journey. periods, as a platform for self-interrogation (ibid 46). Jones takes this further stating that it is artists Although there are elements of self criticism here who push the "boundaries of subjectivity" (2006: the work is also situated in art history, it is post xvii). All motivations are further complicated in an modern so “everything is ok”. This is clearly online exhibition context. Jenny Sundén says that ironic but it nevertheless makes a case for the to exist online, we must continually "write legitimacy of derivative work within post modern ourselves into being" (in Boyd 2006). Online traditions of quotation and inter-textuality. The painters are repeatedly creating and posting new first two responses reassure the artist that the work, in the process reinforcing their artistic and colours are fine. One explicitly references the wider identities. reflexivity of the comments: "The vibrating blue here speaks volumes. And another thing, I like is The Brushes Gallery constitutes a shared display the way you so candidly speak of your process, space for encouraging and displaying what are very enlightening, helps in our own introspection. generally mutually supportive comments. As the ^Up^" (SUZ54241). A discussion follows with the most prolific self-portraitist, Julia L. Kay’s artist who thanks them for “lovely feedback”. Later portfolio (109 images, almost a third of the total in the thread Julia L. Kay adds a comment sample) reveals an uneven spread of diverse comparing BLUEJAYRAY2011's work to her own stylistic treatments and artistic influences. Her previous work and reflection on celebrity output is represented in the images above by her representations of the self. Here then is a copy of one of Matisse's self-portraits. "Julia K’s community of practice where artists comment portrait party" is a group whereby weekly study of reflexively on their own and each other’s work a historical or contemporary artist (e.g. Matisse, both in terms of technique (e.g. colour) and theme Rego, Giacometti) generates new work by (superhero selves). members. This in effect is an "affinity space",

165

14 19 52 59 89 107 Figure 5: BLUEJAYWAY2011 Self Portraits Discussion Further, groups within archives such as Flickr Gunther Kress and Theo van Leeuwen (Kress and enable networks of users to occupy a collective van Leeuwen 2001; Kress, 2008) have discussed curatorial role. However, Kress and van Leeuwen an ideological shift away from economic (2006) describe on the one hand, an inherent "consumption" and associated values towards tension between "authoritative discourses", as human needs. This is evidenced in a move away expressed through rules, assumed knowledge and from the dominance of commodities and explicit and implicit understandings (ibid 12) and capitalism towards design (of products, and also inherent social aspects of creative exchange. This identities). A myriad of digital and networked determines that meanings are generated for resources have been created through open-source interpretation within a particular social group (ibid and commercial activities for autodidacts to 13). It may be assumed that art production and generate and display artifacts and identities. iPad curation groups can now thrive outside the art art practices are enabling a diverse range of establishment. However, content analysis of the individuals — not just the "cultivated" or educated Brushes Gallery Flickr group revealed in the main — to explore aesthetic interests (see Bourdieu conventional and in many cases quite banal forms 1990, on photography), and marginalised artists to of art production. develop a practice (see Doy, also on photography, Towards a conclusion 2005). Writing on technology is very often either While it's recognised that early rhetoric concerning triumphalist or catastrophist (Haas 2012). It would the democratising properties of the internet were be possible to make a triumphalist account for the overstated, the convenience and accessibility of digital painting produced on iPads and archived on the Flickr communities might constitute a Flickr. It could be argued that it democratises not "democratic" space fostering creative production only the production but the dissemination of art. and identity exploration beyond the mainstream art Anyone can now create images which up until world. The spatial constraints of buildings mean relatively recently would have required specialist that any analogue art collection must be exclusive. knowledge and equipment. More than this they The arbiters of taste and cultural gatekeepers such can share this work with other artists, find publics, as curators and collectors have long been criticised even perhaps eventual buyers. Contrariwise for expressing and promoting little more than class catastrophism is equally possible: iPads cost taste (Bourdieu 1986). Online galleries and around four hundred pounds and are then the archives such as the Flickr Brushes group are preserve of the relatively privileged and well off. unlimited by physical space. Facebook is now the Far from offering a new forms of dissemination largest image archive on the planet with some 100 Web 2.0 technologies rather exploit the artists as billion photographs hosted there in mid 2011 producers of free content. (Pingdom 2011). The same source estimates that Dmytri Kleiner and Brian Wyrick recognised early some 4.5 million photographs are uploaded to on how open source principles have been Flickr each day. The 40,000 iPad paintings is then subverted by corporations: a very small subset of the larger image archive but is likely to grow. The image archives are too large investors pocket the value produced by to navigate as a whole. They must be filtered by unpaid users, ride on the technical tags, searches or groups of interest. It is perhaps innovations of the free software movement for this reason that the identity work in evidence in and kill off the decentralising potential of the comments and networks of artists is so peer-to-peer production (2007: 10). important.

166

The critique of Facebook is now also a slogan: if calling all these activities art as long as we keep in you are not paying for the service then you are the mind that such a word may mean very different product. The Flickr service is “free” but users must things in different times and places, and as long as sign up to Yahoo or Google and share information we realise that Art with capital A has no that will enable this company to sell ever more existence” (Gombrich 1985: 4) For both Robert targeted advertising. There are options to upgrade Hughes (1990) and John Berger (1965) the central membership and gain access to more archiving dilemma for the art of the twentieth century was space (which, perhaps enhances an artist's that it lost its power to shock or even to be new "interestingness" ratings?). Members are because it could always be reduced to a generating vast amount of "user generated content" commodity. However shocking the images of for Flickr, and subsequently its parent company, Francis Bacon they could always be reduced to a Yahoo. As David Buckingham (2008) has asked: status symbol on banker’s wall. The new models do such sites promote user generated, or "loser of production and dissemination considered here generated" content? If there is a new economic mean that models of value through scarcity model here it is not for artists but for Yahoo. become impossible. But, in decoupling art from commodity production the digital may enrich What now, is art? visual art at the same time as it impoverishes Triumphalism and Catastrophism are also both artists. options in response to the question of cultural value. As Hockney points out, the technology While the Brushes group artwork may escape the undoubtedly offers new possibilities for art. It trap of becoming a commodity it has not escaped offers infinite layers so colour never becomes the fate of being Flickr content. For Bauman muddied as it would with a watercolour. The Flickr becomes a platform for practices of devices are extremely portable, far more so than "subjectivity fetishism" and self-idealisation (ibid.: sets of travel paints, and so offer the potential for 14–15) whereby a seemingly never ending supply new subjects as well as styles. The facility to draw of artefacts and would-be artists are encouraged to with photographs puts the artist again at the centre explore and display their various selves in visual of representation. Equally, it might be claimed that form. More than twenty years ago Paul Willis the device in fact merely reproduces banality. The (1990) pointed out that theatres, art galleries and marks simulate oils, water colours and other older classical music concerts were of very little interest forms. Digital painting then is nothing but a to the vast majority of the UK population. In Baudrillardian simulacra, an exact copy of contrast to these publicly funded but relatively something that never existed. The iPad is Hilary's elite activities he described "common culture" diaorreah machine, another symptom of the around football, pop music fashion and television "portable confessional" that Zygmunt Bauman (ibid). He argued that such activities did not centre identifies as a means of identity disclosure in around passive consumption, as is often argued, relation to self commodification (2007: 3). His but rather active participation or "grounded dystopian analysis of identity in individualised aesthetics". For Willis people do not watch TV in society — a "modern liquid setting" due to the a passive way, they shout back and argue. A love perceived weakening of human bonds (ibid 49) — of pop music might begin with buying a record but necessitates that one continuously make and end with forming a band. The Flickr groups that remake the self. have formed around iPad apps like Brushes are perhaps similarly engaged in a grounded These concerns are further complicated by cultural aesthetics. Although there is a vast collection of anxiety about what art is and what it is for. work here it receives little to no critical attention. "Signature art" constituting material artifacts Although there are thousands of images viewing produced by the individual artist, in the twentieth figures are for the most part in the tens or century became capital. Its job, as Robert Hughes hundreds. Most publicity around iPad art has pointed out, was to sit on the wall and become centred around established artists like Hockney. more expensive (1990). The form and content of These communities are unconcerned or at least this sort of work were largely irrelevant. As in any ironic about whether or not what they are doing is commodity market what mattered was what people art. They produce and consume their own and each believed it was worth, not what it was. In this others' work, they are their own publics. sense, as EM Gombrich claimed, there is no such thing as art. Although Gombrich is quick to qualify this, noting that there are, however, artists engaged in various activities. “There is no harm in 167

Acknowledgements 18. Gayford, M. A Bigger Message: Conversations Thanks to the artists for permission to use images. with David Hockney. Thames and Hudson, The research has been funded by the EPSRC. London, 2007. 19. Gee, J. (2004) Situated Language and Learning: A References Critique of Traditional Schooling. New York: 1. Adami, E. (2008). 'Learning to (You)Tube () the Routledge. pattern of gaze'. Presentation: International 20. Gombrich, E. M. (1985) The Story of Art. Phaidon Conference on Multimodality and Learning, 19- Press Limited. 20th June Institute of Ed., University of London. 21. Guggenheim Bilbao website: 2. Appleyard B. (2011) David Hockney Interview http://www.guggenheim.org/bilbao/exhibitions/dav [online] http://www.bryanappleyard.com/david- id-hockney-a- bigger-picture hockney-interview/ (accessed 12.06.12) 22. Haas L., (2012) A More Crocodile Crocodile. 3. Bardzell, J (2007) Creativity in Amateur London Review of Books. Vol 34. No 4. Multimedia: Popular Culture, Critical Theory, and 23. Howells, R. (2003) Visual Culture. Cambridge: HCI. In: Human Technology, 3 (1), February 2007, Polity. 12–33. 24. Hughes R., (1990) The Shock of the New: Art and 4. Bauman, Z. (2007) Consuming Life Cambridge: the Century of Change. Thames and Hudson. Polity. 25. Jones, A. (2006) Self/Image: Technology, 5. Berger J (1965) Success and Failure of Picasso. representation, and the contemporary subject. Penguin Books. Abingdon, Oxon: Routledge. 6. BLUEJAYWAY website: 26. Kleiner, D. and Wyrick, B. (2007) 'Info-Enclosure www.bluejaywaydigitalarts.co.uk/about-us.php 2.0'. Mute, 2 (4) January, pp. 14–19. (accessed 17th June 2012). 27. Kress, G. (2008) New Literacies, New 7. Blush on YouTube: Democracies. (Beyond Current Horizons) [Online]. http://www.youtube.com/watch?feature=player_em Available: bedded&v=vnWWp-Mac_E (accessed 14.06.12). http://www.beyondcurrenthorizons.org.uk/wp- 8. Bourdieu, P. (1984) Distinction. Abingdon: content/uploads/ Routledge (and Paul Kegan). bch_challenge_paper_democracies_gunther_kress2 9. Bourdieu, P. (1990) Photography: A middle-brow .pdf (accessed 14.06.12). art. Cambridge: Polity Press. 28. Kress, G. and van Leeuwen, T. (2001) Multimodal 10. Boyd, D. (2006) 'Friends, Friendsters, and Discourse: The Modes and Media of Contemporary MySpace Top 8: Writing Community Into Being Communication, London: Arnold. on Social Network Sites.' First Monday 11:12, 29. Kress, G. and van Leeuwen, T. (2006). Reading December [online]. Available from: Images: The Grammar of Visual Design Abingdon, http://firstmonday.org/htbin/ Oxon, Routledge. cgiwrap/bin/ojs/index.php/fm/article/view/1418/13 30. Madison, J. (2011) Daily Mail article on Shawn 36 (accessed15.06.12). Harris's video: http://www.dailymail.co.uk/news/ 11. Brushes website: http://www.brushesapp.com/ article-2007357/Blush-music-video-featuring- (accessed 15.06.12). Snoop-Dogg-created-iPad-Brushes-app-Shawn- 12. Briggs, J. (2010). Visual art practices: digital Harris.html literacies and the construction of identities in 31. Mayring, P. (2004). "Qualitative Content Analysis Northern Ireland. Ph.D, University of Ulster, in Flickr". In: U. Kardorff and E. Steinke, (Eds.) A Belfast. Companion to Qualitative Research. Sage, London. 13. Buckingham, D. (2007) Beyond Technology: 32. Pingdom (2011) Children's learning in the age of digital culture. http://royal.pingdom.com/2012/01/17/internet- Cambridge: Polity. 2011-in-numbers/ 14. Buckingham, D. (2008) Conference Keynote In: 33. Royal Academy website: proceedings: Digital Content Creation: Creativity, http://www.royalacademy.org.uk/exhibitions/hockn Competence, Critique, DREAM, University of ey/ (accessed 14.06.12). Southern Denmark, 18–20 Sept, Odense, Denmark. 34. Wesley Hein blog on Flickr 'interesting' algorithm 15. Burn, A. and Parker, D. 2003. 'Tiger's Big Plan: http://www.webdesignerdepot.com/2010/08/master Multimodality and the Moving Image'. In: G. Kress ing-flickr-a-practical-guide/ (accessed 14.06.12). and C. Jewitt (eds.) Multimodal Literacy. New 35. Willis, P. E. (1990). Common Culture: Symbolic York: Peter Lang, pp. 56–72. work at play in the everyday cultures of the young, 16. Digital Originals http://di.ncl.ac.uk/blog/digital- Milton Keynes, Open University Press. originals/ 36. Wikipedia, iPad : http://en.wikipedia.org/wiki/IPad 17. Doy, G. (2005) Picturing the Self: Changing views 37. Wikipedia, Flickr: of the subject in visual culture. London: I.B. http://en.wikipedia.org/wiki/Flickr. Taurus.

168