DOCSLIB.ORG

Interactive Audiovisual Objects

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Interactive Audiovisual Objects This dissertation presents four interactive audiovisual pro- jects by Video Jack, a duo composed of Nuno N. Correia and Interactive André Carrilho. The projects are: Heat Seeker (2006), AVOL Audiovisual (2007), Master and Margarita (2009) and AV Clash (2010). The three last works have adopted an Interactive AudioVisual Ob- Objects jects (IAVO) approach, consisting of the integration of sound, audio visualization and graphical user interface into modular 21/2013 Aalto-DD units. The projects have been developed with a crossmedia perspective, ranging from performance to installation and net art, and the study emphasizes the last area. The conclusions Nuno N. relate to audiovisual content, interaction design and user ex- Correia perience. The IAVO approach is presented as a path to create projects for integrated audiovisual art that are coherent, flex- ible, easy to use, playful and engaging to experience. Interactive Audiovisual Objects ISBN 978-952-60-5000-3 BUSINESS + ISBN 978-952-60-5001-0 (pdf) ECONOMY ISSN-L 1799-4934 ISSN 1799-4934 ART + ISSN 1799-4942 (pdf) DESIGN + ARCHITECTURE Nuno N. Correia Aalto University School of Arts, Design and Architecture SCIENCE + Department of Media TECHNOLOGY books.aalto.fi www.aalto.fi CROSSOVER DOCTORAL DISSERTATIONS Interactive Audiovisual Objects Nuno N. Correia Nuno N. Correia Interactive Audiovisual Objects Aalto University publication series DOCTORAL DISSERTATIONS 21/2013 School of Arts, Design and Architecture Aalto ARTS Books Helsinki books.aalto.fi © Nuno N. Correia Graphic design: Hannu Hirstiö Cover: Image from AV Clash Materials: Chromolux 275g, Munken Print White 115g Font: Helvetica Neue ISBN 978-952-60-5000-3 ISBN 978-952-60-5001-0 (pdf) ISSN-L 1799-4934 ISSN 1799-4934 ISSN 1799-4942 (pdf) Printed in Unigrafia Helsinki 2013 3.4.3 Tool or soundtoy? 75 Table of Contents 3.4.4 Different segments detected 76 3.4.5 Summary regarding experience 77 3.5 Project management 78 Summary 8 3.5.1 Cross-platform spin-offs 78 3.5.2 Promotion 81 Acknowledgements 9 3.5.3 Documentation 83 3.5.4 Technical issues 83 1 Introduction 13 3.5.5 Copyright issues 83 1.1 Structure of the dissertation 15 3.6 Methodology 84 1.2 Research questions and topics 16 3.7 Future developments 86 1.3 Methodology 18 3.8 Contributions 90 1.4 Publications and practical work 21 1.4.1 Publications 21 4 References 94 1.4.2 Interactive AudioVisual Objects (IAVO) approach 23 1.4.3 Projects 26 List of Figures 100 1.4.4 Web integration 32 1.5 Background, motivation and related work 33 5 Article 1: Heat Seeker – an Interactive Audio-Visual Project 1.5.1 Early interest in music and programming 33 for Performance, Video and Web 104 1.5.2 Initial work with André Carrilho 34 1.5.3 Other Video Jack work following Heat Seeker 36 6 Article 2: AVOL – Towards an Integrated Audio-Visual Expression 118 1.5.4 Other relevant work 38 7 Article 3: Master And Margarita – an Audiovisual Adaptation of 2 Contextualization: sound, image and interaction 42 Bulgakov’s Novel for the Web and Performance 132 2.1 Historical context and main influences in audiovisual art 42 2.2 Related concepts 48 8 Article 4: AV Clash – Online Tool for Mixing and Visualizing 2.2.1 Visual music and audiovisual 48 Audio Retrieved from Freesound.org Database 140 2.2.2 VJing 48 2.2.3 Synesthesia 50 9 Article 5: Master and Margarita: from Novel to Interactive 2.3 Image and sound in cinema, music video and performances 53 Audiovisual Adaptation 152 2.3.1 Cinema and “music film” 53 2.3.2 Music video 54 10 Article 6: AV Clash, Online Audiovisual Project: 2.3.3 Music performances 55 a Case Study of Evaluation in New Media Art 168 2.4 Audiovisual mobile apps 56 2.5 Further reflections into audio-vision 58 11 Article 7: Paths in Interactive Sound Visualization: 2.5.1 Inherent multi-sensoriality in music 58 from AVOL to AV Clash 186 2.5.2 Audio-Vision 59 2.6 Audio-vision and interaction 61 12 Annexes 200 2.6.1 Visual feedback in audio software 61 2.6.2 Design of audiovisual systems 62 Annex 1 List of performances and presentations 201 3 Conclusions 66 Annex 2 Guide to web content and documentation 202 3.1 IAVO approach and project development path 66 3.2 Content 69 Annex 3 AV Clash web statistics 205 3.2.1 Sonic and visual content 69 3.2.2 Adaptation 70 Annex 4 Questionnaire 210 3.2.3 Integration of sound and visuals 71 3.2.4 Vectors and loops 72 Annex 5 Summary of the answers to the questionnaire 220 3.3 Interactivity 72 3.3.1 Interactivity versus automatism and linearity 72 3.3.2 IAVO approach and flexibility 73 3.3.3 Ease of use and explorability 73 3.4 Experience 74 3.4.1 Engagement and playfulness 74 3.4.2 Creativity and expression 75 8 Summary Acknowledgements 9 This dissertation studies four projects combining visuals, I would like to thank Tapio Takala and Sergi Jordà (my su- sound and interactivity by the author and André Carrilho (un- pervisors), Lily Diaz, Philip Dean, Rasmus Vuori, Koray Tahi- der the name Video Jack). The aim of the study is to create roglu, Antti Ikonen, Markku Reunanen, Gokce Taskan, Nuno web-based interactive audiovisual art projects for integrated Correia (from Universidade Nova de Lisboa), Leena Eilittia, and simultaneous manipulation of sound and motion graph- Anneli Noor, Miguel Espada, and the Media Lab Helsinki ics, in a way that is coherent, flexible, easy to use, playful, community in general, for their support and advice. I would and engaging to experience. From this study aim, three major also like to acknowledge the reviewers of my writings, in par- research topics emerge: content, interactivity and experience. ticular the pre-examiners, Todd Winkler and Mick Grierson, for their feedback. I am also grateful to the graphic designer The four projects included in the study are: Heat Seeker of this dissertation, Hannu Hirstiö, Sanna Tyyri-Pohjonen for (2006), AVOL (2007), Master and Margarita (2009) and AV the support in the publication, and to the questionnaire re- Clash (2010).1 The three last projects have adopted an In- spondents. A special thank you to André Carrilho for his ar- teractive AudioVisual Objects (IAVO) approach, introduced in tistic collaboration, without which this work would not have this study, consisting of the integration of sound, audio visu- been possible. alization and Graphical User Interface (GUI). Following this approach, GUI elements are embedded in the visuals, and During these years, I have performed and presented my pro- aesthetically integrated with the animation style and with the jects in several festivals and events. I would like to express overall visual character of each project. These projects have my gratitude to all the event organizers, curators, promoters, been developed with a cross-platform perspective (perfor- bloggers, audiences and fellow artists, for the opportunity mance, net art, non-interactive video, soundtrack and exhi- to show my work, and also for the constructive discussions bition), and the study focuses on the net art versions. (offline and online). I would also like to thank the institutions who have provided me with the funding that supported this The methodology for the dissertation is practice-based re- work: Portugal’s Fundação para a Ciência e Tecnologia and search, complemented by a user study. The background and Aalto University, School of Arts, Design and Architecture. Ad- motivation for the work are presented, and the projects are ditionally, I would like to thank the Institute of Informatics of contextualized with related works. The study is framed within Tallinn University for hosting me as a guest researcher. the field of audiovisual art, and connections are established between the projects and related concepts. The combination Finally, I would like to thank my parents for their constant of audio, visuals and interaction is discussed. support and encouragement. The conclusions are grouped around six topics: content, Helsinki, January 2013 interactivity, experience (the main research topics), pro- ject management, methodology, and future developments. Nuno N. Correia Strengths and weaknesses detected in the projects are ana- lyzed, taking into account the results of the user study. These are followed by more generic conclusions that aim to provide useful contributions to the field of interactive audiovisual art. 1 The projects can be accessed from: htttp://www.videojackstudios.com 10 11 Heat Seeker Image from Image from 12 1 Introduction 13 “To remove the barriers between sight and sound, between the seen world and the heard world! To bring about a unity and a harmonious relationship between these two opposite spheres. What an absorbing task! The Greeks and Diderot, Wagner and Scriabin – who has not dreamt of this ideal? Is there anyone who has made no attempt to realize this dream?” (Eisenstein 1986) This dissertation studies four projects combining audio, visu- als and interactivity that I have developed with visual artist André Carrilho. We have released these projects under the alias “Video Jack”. The aim of the study is to create web- based interactive audiovisual art projects for integrated and simultaneous manipulation of sound and motion graphics, in a way that is coherent, flexible, easy to use, playful, and en- gaging to experience. The projects developed in the scope of the study were: Heat Seeker (2006), AVOL (2007), Master and Margarita (2009) and AV Clash (2010). The projects have been developed with a cross-platform perspective, having been presented in the following formats: performance, net art, non-interactive video, soundtrack and exhibition (ex- hibited as interactive installation in the case of AVOL and AV Clash; as video screening in the case of Heat Seeker; Master and Margarita has not been presented as exhibition).
Load more

Recommended publications
  • THE AUDIOVISUAL BREAKTHROUGH Ana Carvalho
    THE AUDIOVISUAL BREAKTHROUGH Ana Carvalho and Cornelia Lund (eds.) 21 41 83 109 129 THE AUDIOVISUAL BREAKTHROUGH 7 PRACTICE AND DISCOURSE. AN INTRODUCTION AS MANUAL Ana Carvalho and Cornelia Lund The Audiovisual Breakthrough guides us across the landscape of artistic live practi- ces that present sound and image through technological means. This landscape has been radically reshaped during the last 20 years due to technological developments causing what we might call an “audiovisual breakthrough,” which means that audio- visual artistic production has gained a certain visibility and a certain, even institutionalized, standing. The main objective of this book, however, is not to portray this landscape with its main players and their activities, but to find out more about the underlying concepts that help us explain these activities. � Whoever has been trying to write an academic or curatorial text on this area has probably felt trapped in a confusing web of unclear, or even inconsistent, definitions. Visual music, expanded cinema, VJing, live cinema, and live audiovisual performance are the most widely used concepts here, each of these terms addressing a different angle of contemporary audiovisual pro- duction contextualized within specific fea- tures and a related history. Holding this in mind, The Audiovisual Breakthrough aims at developing useful definitions for both the theoretical debate and the performance obvious that clarifications were needed for context. � � � We might of course say— meaningful communication about and with- especially as performers—that we “really 9 in the field of artistic AV production to be don’t care” and that we are “more interested possible in the future.
    [Show full text]
  • 2016-Program-Book-Corrected.Pdf
    A flagship project of the New York Philharmonic, the NY PHIL BIENNIAL is a wide-ranging exploration of today’s music that brings together an international roster of composers, performers, and curatorial voices for concerts presented both on the Lincoln Center campus and with partners in venues throughout the city. The second NY PHIL BIENNIAL, taking place May 23–June 11, 2016, features diverse programs — ranging from solo works and a chamber opera to large scale symphonies — by more than 100 composers, more than half of whom are American; presents some of the country’s top music schools and youth choruses; and expands to more New York City neighborhoods. A range of events and activities has been created to engender an ongoing dialogue among artists, composers, and audience members. Partners in the 2016 NY PHIL BIENNIAL include National Sawdust; 92nd Street Y; Aspen Music Festival and School; Interlochen Center for the Arts; League of Composers/ISCM; Lincoln Center for the Performing Arts; LUCERNE FESTIVAL; MetLiveArts; New York City Electroacoustic Music Festival; Whitney Museum of American Art; WQXR’s Q2 Music; and Yale School of Music. Major support for the NY PHIL BIENNIAL is provided by The Andrew W. Mellon Foundation, The Fan Fox and Leslie R. Samuels Foundation, and The Francis Goelet Fund. Additional funding is provided by the Howard Gilman Foundation and Honey M. Kurtz. NEW YORK CITY ELECTROACOUSTIC MUSIC FESTIVAL __ JUNE 5-7, 2016 JUNE 13-19, 2016 __ www.nycemf.org CONTENTS ACKNOWLEDGEMENTS 4 DIRECTOR’S WELCOME 5 LOCATIONS 5 FESTIVAL SCHEDULE 7 COMMITTEE & STAFF 10 PROGRAMS AND NOTES 11 INSTALLATIONS 88 PRESENTATIONS 90 COMPOSERS 92 PERFORMERS 141 ACKNOWLEDGEMENTS THE NEW YORK PHILHARMONIC ORCHESTRA THE AMPHION FOUNDATION DIRECTOR’S LOCATIONS WELCOME NATIONAL SAWDUST 80 North Sixth Street Brooklyn, NY 11249 Welcome to NYCEMF 2016! Corner of Sixth Street and Wythe Avenue.
    [Show full text]
  • EMAF Katalog 2016
    SPECIALS FMB ANNIVERSARY & FILMFESTSPEZIAL-TV 30 Jahre Film & Medienbüro Niedersachsen – Perspektiven für Förderung von Medienkunst? Der frisch gewählte erste Vorstand des Film & Medienbüros: Jochen Coldewey, Henning Kunze, Susanne Degener und Marianne Lohmann. Nicht auf dem Foto: Gerd Dahlmann und Andreas Franzen. Vor 30 Jahren, am 31. Mai 1986, wurde das Film & Medienbüro Niedersachsen (FMB) während des In- ternationalen Experimentalfilmworkshop, dem Vorläufer des EMAF, in Osnabrück gegründet. Deshalb wird es am 22.4.2016 um 17 Uhr eine Veranstaltung beim EMAF geben, bei der wir mit Jochen Colde- wey, dem 1986 gewählten 1. Vorsitzenden des FMB, Michael Brynntrup (HBK Braunschweig), Alexandra Gerbaulet (Regisseurin von ›Schicht‹) und weiteren Gästen nicht nur über alte Zeiten und die Schwierig- keiten, von Experimentalfilm oder Medienkunst leben und überleben zu können sprechen wollen. Es soll auch um Perspektiven gehen, was kann strukturell verbessert werden, welche neuen Plattformen und Verbreitungswege für Medienkunst gibt es und wie können die sinnvoll genutzt werden. Und: welche Be- deutung hat die Filmförderung. Infos: www.filmbuero-nds.de. FilmFestSpezial vom EMAF Das Festival-TV-Magazin berichtet in einer 45-minütigen Sendung über das EMAF. Interviews mit Künst- lern und Jurymitgliedern, Filmausschnitte, Einblicke in die Workshops und die Ausstellung sollen die spannende Vielfalt des Festivals einfangen. Die EMAF-Sendung ist ab 4. Mai 2016 bei 11 Bürger-TV-Sen- dern – zum Teil als Stream – zu sehen: h1 Hannover, oeins Oldenburg, TV 38 Wolfsburg/Braunschweig, Radio Weser TV Bremen, Tide-TV Hamburg, Alex Berlin, OK Kiel, OK Kassel, rok-tv Rostock, OK Flens- burg und Fernsehen aus Schwerin. Die einzelnen Beiträge und Interviews sind bereits vorher auf www.youtube.com/ffspezial zu sehen.
    [Show full text]
  • Noordegraaf DEF.Indd 1 08-01-13 14:33 FRAMING FILM
    FRAMING ET AL. (EDS.) NOORDEGRAAF JULIA NOORDEGRAAF, COSETTA G. SABA, FILM BARBARA LE MAÎTRE, VINZENZ HEDIGER (EDS.) PRESERVING AND EXHIBITING MEDIA ART PRESERVING AND EXHIBITING MEDIA ART Challenges and Perspectives This important and fi rst-of-its-kind collection addresses the Julia Noordegraaf is AND EXHIBITING MEDIA ART PRESERVING emerging challenges in the fi eld of media art preservation and Professor of Heritage and exhibition, providing an outline for the training of professionals in Digital Culture at the Univer- this fi eld. Since the emergence of time-based media such as fi lm, sity of Amsterdam. video and digital technology, artists have used them to experiment Cosetta G. Saba is Associate with their potential. The resulting artworks, with their basis in Professor of Film Analysis rapidly developing technologies that cross over into other domains and Audiovisual Practices in such as broadcasting and social media, have challenged the tradi- Media Art at the University tional infrastructures for the collection, preservation and exhibition of Udine. of art. Addressing these challenges, the authors provide a historical Barbara Le Maître is and theoretical survey of the fi eld, and introduce students to the Associate Professor of challenges and di culties of preserving and exhibiting media art Theory and Aesthetics of through a series of fi rst-hand case studies. Situated at the threshold Static and Time-based Images at the Université Sorbonne between archival practices and fi lm and media theory, it also makes nouvelle – Paris 3. a strong contribution to the growing literature on archive theory Vinzenz Hediger is and archival practices. Professor of Film Studies at the Goethe-Universität Frankfurt am Main.
    [Show full text]
  • Media Art in Central and Eastern European Countries - an Outline
    Ryszard W. Kluszczynski The Past and Present of (Multi)Media Art in Central and Eastern European Countries - An Outline [3.983 words] Poland Ryszard W. Kluszczynski The Past and Present of (Multi)Media Art in Central and Eastern European Countries - An Outline This text was archived at the Institute for Contemporary Art in Zagreb collection, as part of the Research project conceived in 1997 by a SCCAN – Soros Centers for Contemporary Art Network, funded by the Open Society Foundation, New York. The purpose of the project was to select, collect and disseminate texts on contemporary art practices in the Central and Eastern Europe and Central Asia, around Soros Centers for Contemporary Art, written in and about art of the 1990s. The coordination of the project was carried out by Janka Vukmir, SCCA – Zagreb, today the Institute for Contemporary Art, Zagreb. We did not intervene in any of texts more than just correcting obvious typos and spelling. On the occasion of collecting texts, we were given permission from all authors, to rightfully use them. If anyone now has different instructions, please, contact us at the [email protected]. All of the texts we have collected at the time have been later published on the website of the I_CAN, International Contemporary Art Network, the short-lived successor of the SCCAN. On the occasion of the exhibition 90s: Scars, revisiting the art practices and social and political context of the 1990s in the postcommunist countries, the Institute for Contemporary Art is now reoffering a collection of 89 texts and a comprehensive list of then proposed further readings, on the website of the Institute for Contemporary Art, www.institute.hr.
    [Show full text]
  • Front Matter
    Copyright by Leonardo de Cardoso 2010 The Report Committee for Leonardo de Cardoso Certifies that this is the approved version of the following report: Visual Music: an Ethnography of an Experimental Art in Los Angeles APPROVED BY SUPERVISING COMMITTEE: Supervisor: Veit Erlmann Sonia Seeman Visual Music: an Ethnography of an Experimental Art in Los Angeles by Leonardo de Cardoso, B.M. Report Presented to the Faculty of the Graduate School of Music The University of Texas at Austin in Partial Fulfillment of the Requirements for the Degree of Master’s of Music Master of Music The University of Texas at Austin May 2010 Dedication To my parents, Edmundo and Arminda. Acknowledgements My idea with this study was to write a narrative where voices other than mine could be put together, and where the resulting consonances and dissonances could somehow delineate the field I was trying to understand. In that sense, I want to thank all my ethnographic collaborators for being part of this research. Hopefully I have not misrepresented any of them, and my own interpretation of their practices will be understood as a valid and thought-provoking point of view. Other collaborators I would like thank include ethnomusicologists from the Group of Musical Studies at the Federal University of Rio Grande do Sul, Brazil, with whom I was associated from 2005 to 2008 and where I found a fruitful space to learn collaboratively about anthropology, musicology, and ethnomusicology. At the University of Texas at Austin I was also nurtured with stimulating ideas. Professors James Buhler, Caroline O’Meara, and Robin Moore assisted me in different ways during my studies.
    [Show full text]
  • The Academy of the Senses
    The Academy of the Senses SYNESTHETICS IN SCIENCE, ART, AND EDUCATION FRANS EVERS The Academy of the Senses SYNESTHETICS IN SCIENCE, ART, AND EDUCATION FRANS EVERS Compiled and edited by VINCENT W.J. VAN GERVEN OEI ArtScience Interfaculty Press SYN·ES·THET·ICS n. 1. (used with a sing. verb) a. The branch of esthetics (Eng. aesthetics) dealing with joint esthetics, the artistic effects resulting from bringing together esthetics from different disciplines in one work of art, often as a consequence of the use of new technologies b. In Kantian philosophy esthetics is the branch of metaphysics concerned with the laws of perception. Synesthesia “is a term that refers to the transposition of sensory images or sensory attributes from one modality to the other”(Marks). The word synesthesia is composed of two elements: syn (with, together, alike, similarity) and aesthesia (to feel, per- ceive). In analogy we may describe synesthetics as “joint esthetics” c. Synesthetics, synesthetic art, verbal synesthesia, and sensory synesthesia are all manifestations of a guiding perceptual principle: “the unity of the senses.” According to ethnomusicologist Erich M. von Hornbostel this interrelatedness of the senses can be observed in daily life situations, in new media such as film as well as in the unity of the arts which was given from the origin (masked dance) 2. (used with a sing. verb) The study of the esthetics of visual music (Castel), Gesamtkunst (Wagner, Kandinsky), the art of relationships (Mondrian, Moholy-Nagy), synaesthetics and kinaesthetics: the way of all experience (Youngblood) as well as more recent forms of genera- tive art, interactive art and mediated environments 3.
    [Show full text]
  • Visual Music Masters Adriano Abbado
    Visual Music Masters Adriano Abbado Visual Music Masters Abstract Explorations: History and Contemporary Research Contents Cover The author wishes to thank: 7 Introduction Bret Battey Luciana Abbado Pestalozza and Clonal Colonies, detail, Paolo Martelli for their invaluable I. History 2011 support; Costanza Tessarolo and Marcello Abbado for their 10 1 The Beginnings Design suggestions; Melinda Mele for 18 2 Music and Early Twentieth-Century Painting Marcello Francone her review of the original Italian 28 3 Luminous Instruments Copy Editor text; Emmanuel Gallina, 42 4 Abstract Films Maria Conconi Elisabetta Minonne, Rembert 66 5 Music and Abstract Art Layout Noppe, the Prometheus Centre, Anna Cattaneo Andreas Seegatz and Dagmar 76 6 Electronic Art Trinks for their help regarding Iconographic Research Paola Lamanna foreign language issues. II. Contemporary Developments Thank you also to the publisher Translation 90 7 Animations and Videos Skira and to all the artists and Melinda Mele 106 8 Installations organizations that have kindly contributed to the making of this 124 9 Performances book. 136 10 Language All rights reserved under international copyright Appendix conventions. No part of this book may be reproduced or utilized 142 Synesthesia in any form or by any means, electronic or mechanical, References including photocopying, 150 Bibliography recording, or any information storage and retrieval system, 156 Websites without permission in writing 168 Index of Names from the publisher. 175 Photo credits © Giacomo Balla, Vladimir Baranoff-Rossiné, Hans Richter, Leopold Survage, Jean Tinguely, by SIAE 2017 © Antonella Vigliani Bragaglia, by SIAE 2017 © Succession Marcel Duchamp, by SIAE 2017 © Man Ray Trust, by SIAE 2017 © 2017 Skira editore, Milano First published in Italy in 2017 by Skira Editore S.p.A.
    [Show full text]