Movement and Sound: Intersecting Art Practices with HCI, Multimodal Interfaces and Social Interaction
Total Page:16
File Type:pdf, Size:1020Kb
Movement and Sound: intersecting art practices with HCI, multimodal interfaces and social interaction. Frédéric Bevilacqua Sound Music Movement Interaction Team Ircam - Centre Pompidou STMS Ircam-CNRS-UPMC http://ismm.ircam.fr Institut de Recherche et Coordination Acoustique/Musique http://www.ircam.fr Music and Sciences and Technologies Performing Arts for Sound and Music Art works Scientific research Music compositions Technology development Dance performances Industrial partnerships Installations Computation Culture Performance Modelling Creativity Arts Signal processing Music technology Cognition Human Computer Emotion Interaction Embodiment Expressivity Perception Machine learning Motor control Music Technology in Europe • Digital Music Industries : ‣ growing revenues in Europe (important role of SMEs) ‣ Music Software for music creation • Native Instruments • Ableton ‣ Music distribution • Spotify • Deezer • Soundcloud IFPI report, 2013. www.ifpi.org/downloads/dmr2013-full-report_english.pdf symbolic symbolic I’ll be there ok :) signal signal ok :( movement movement non-verbal non-verbal communication “communication” Human communication vs Music symbolic I’ll be there Speech recognition Music Information Retrival audio audio signal multimodal movement Affect, Emotion, Expressivity interfaces etc.. recognition Motion Sound Interactions performers concert musicians installation ...and non-musicians social event game sound Mediated capture Interaction gesture capture Computer-aided Improvisation OMax and SoMax G. Assayag et al., Music Representations Team , Ircam, http://repmus.ircam.fr/omax/home Music Improvisational Agents vs Conversational Agents • Is there an Adaptive Interaction Model that could work for both ? ‣ turn taking / synchronization ‣ anticipation ‣ affects ... PhD Work K Sanlaville, Collaboration Ircam-Telecom ParisTech (Assayag, Bevilacqua, Pelachaud) Digital Musical Instruments Multimodal Interfaces for (Musical) Expression Biomuse, Musical Instrument- 20th A. Tanaka et al., 1992 Kar Very Nervous System David Rokeby (1990) Smule - iphone The Hands, M. Waisvisz, 1984 Theremin, L. Theremin, 1928 ReacTable, saxophone S. Jorda et al., 2005 Electric Guitar EEG and Music Alvin Lucier, 1965 1846 20th ’20 21th MO – Modular Musical Objects 1st prize at the 2011 Margaret Guthman Musical Instrument Competition International Design Biennial, St-Etienne 2010 Talk to Me, MoMA – NY 2011 Award “Societal Impact” from the French National Research Agency one object, one use objects to assemble active object and accessories hacking other objects and uses Modular Musical Objects IRCAM, NoDesign, Da Fact 2011 https://www.youtube.com/watch?v=Uhps_U2E9OM The MO-Kitchen IRCAM, 2011 Modular Musical Objects IRCAM, NoDesign, Da Fact 2011 https://www.youtube.com/watch?v=Uhps_U2E9OM Mogees IRCAM, Goldsmiths, 2011/2012 DIRTI, Dirty Tangible Interfaces, User Studio, IRCAM, R. Cahen 2012 (projet Topophonie) http://en.userstudio.fr/projects/dirti-for-ipad/ DirTI, Dirty Tangible Interfaces User Studio, IRCAM, R. Cahen 2012 http://en.userstudio.fr/projects/dirti-for-ipad/ Interlude IRCAM, Atelier des Feuillantines, NoDesign, GRAME et al. 2011 https://www.youtube.com/watch?v=Uhps_U2E9OM MO IRCAM, NoDesign, GRAME et al. 2011 https://www.youtube.com/watch?v=Uhps_U2E9OM Urban Musical Game – Early Tests IRCAM, NoDesign, Phonotonic, Andrea Cera, 2011 https://vimeo.com/22120867 Urban Musical Game – Festival Futur en Seine IRCAM, NoDesign, Phonotonic, Andrea Cera 2011 https://vimeo.com/63974510 Phonotonic, Interactive Music Battle http://www.phonotonic.net/ Why is this important ? • Concepts ‣ Collaborative interaction (mediated interaction) ‣ “Re-using known” gestures - skill transfer • Important applications, potential societal impact ‣ Serious Gaming ‣ Pedagogy: sport and music ‣ Rehabilitation Non verbal communication ....using sounds Francoise, Schnell, Bevilacqua, Siggrah 2014, Emerging Technologies Using muscle signals (EMG) MuBu Funk Scat Share Atau Tanaka, IRCAM, 2012 Collaborative Situated Media Or Wrap-up • Work with artists and designers ! and creative industries ! • Convergence ‣ Multimodal interfaces ‣ Social Interaction • Don’t forget the sound ‣ Music ‣ Sound as feedback mechanism (sonification) • Towards participative concerts and events IRCAM – Real-Time Musical Interactions Frédéric Bevilacqua, Julien Bloit, Riccardo Borghesi, Eric Boyer, Baptiste Caramiaux, Sarah Fdili Alaoui, Emmanuel Fléty, Jules Françoise, Jean-Philippe Lambert, Fabrice Guédy, Nicolas Rasamimanana, Norbert Schnell, Diemo Schwarz, Bruno Zamborlin with also... Interlude consortium GRAME, NoDesign, Atelier des Feuillantines, Da Fact Phonotonic Sylvain Hanneton Université Paris Descartes, Agnès Roby-Brami UPMC Topophonie partners ENSCI, Orbe, UserStudio Goldsmiths University of London Pierre Jodlowski, Andrea Cera, Roland Cahen ... and the support of ANR, EU-IST, Région Ile de France, Cap Digital.