MUSIC COMPUTING GROUP DEPARTMENT OF COMPUTING AND COMMUNICATIONS FACULTY OF MATHEMATICS,COMPUTING AND TECHNOLOGY THE OPEN UNIVERSITY Tabletop Tangible Interfaces for Music Performance: Design and Evaluation Author: Anna Xambó Sedó MSc (Universitat Pompeu Fabra) BA, MA (Universitat de Barcelona) A thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy Supervisors: Robin Laney (The Open University, UK) Chris Dobbyn (The Open University, UK) Sergi Jordà Puig (Universitat Pompeu Fabra, Spain) Examiners: Eduardo Reck Miranda (Plymouth University, UK) Janet van der Linden (The Open University, UK) Submitted 30 September 2014 Abstract This thesis investigates a new generation of collaborative systems: tabletop tangi- ble interfaces (TTIs) for music performance or musical tabletops. Musical tabletops are designed for professional musical performance, as well as for casual interaction in public settings. These systems support co-located collaboration, offered by a shared interface. However, we still know little about their challenges and opportunities for collaborative musical practice: in particular, how to best support beginners or ex- perts or both. This thesis explores the nature of collaboration on TTIs for music performance be- tween beginners, experts, or both. Empirical work was done in two stages: 1) an exploratory stage; and 2) an experimental stage. In the exploratory stage we studied the Reactable, a commercial musical tabletop designed for beginners and experts. In particular, we explored its use in two environments: a multi-session study with expert musicians in a casual lab setting; and a field study with casual visitors in a science centre. In the experimental stage we conducted a controlled experiment for mixed groups using a bespoke musical tabletop interface, SoundXY4. The design of this study was informed by the previous stage about a need to support better real- time awareness of the group activity (workspace awareness) in early interactions. For the three studies, groups musical improvisation was video-captured unobtrusively with the aim of understanding natural uses during group musical practice. Rich video data was carefully analysed focusing on the nature of social interaction and how workspace awareness was manifested. The findings suggest that musical tabletops can support peer learning during multi- ple sessions; fluid between-group social interaction in public settings; and a demo- cratic and ecological approach to music performance. The findings also point to how workspace awareness can be enhanced in early interactions with TTIs using auditory feedback with ambisonics spatialisation. The thesis concludes with theoretical, methodological, and practical implications for future research in New Interfaces for Musical Expression (NIME), tabletop studies, and Human-Computer Interaction (HCI). Acknowledgements I owe each of my supervisors, Robin Laney, Chris Dobbyn, and Sergi Jordà, a debt of gratitude for their constant and invaluable help, enthusiasm, patience, support, and guidance during this research. Thanks to Robin Laney for giving me the opportunity to first come at the OU as a visitor researcher, and to then pursue a PhD degree, and for always being there. Thanks to Chris Dobbyn for his philosophical enquiries, for constantly highlighting the importance of the precision of language, and for helping me to substantially improve my writing. Thanks to Sergi Jordà for always making me think differently, for endlessly sharing his knowledge in the last nine years, and for continuous thoughtful discussions on this research. My gratitude to the excellent academic environment at the OU. I am truly indebted with Eva Hornecker and Paul Marshall for introducing me to research on interac- tion analysis, and for our fabulous video analysis sessions. I am grateful to the members of the OU Computing Music Lab for lively intellectual discussions, espe- cially and with no particular order, Andrew Milne, Rose Johnson, Katie Wilkie, Tom Collins, Simon Holland, Andrea Franceschini, Stella Paschalidou, Anthony Prechtl, and Adam Linson. Special thanks to Marian Petre, Sheep Dalton, Karen Littleton, Clara Mancini, Antonio González-Torres, Martyn Hammersley, John Oates, Ruth Neil, Mariano Rico, Alistair Willis, Debbie Briggs, Mary McMahon, Danielle Lilly, Catherine McNulty, and all who I forget, for their ideas, help, advice and support during this journey. Thanks to my probation examiners Yvonne Rogers and Hugh Robinson for offering constructive criticism. Thanks to my mock viva examiners Si- mon Holland and Allan Jones for their useful advice. Thank you to my viva exam panel chair Laurence Dooley for chairing the session, and my viva examiners Ed- uardo Miranda and Janet van der Linden for their invaluable feedback. I wish to thank all the participants of our studies, as this research would not have been possible otherwise. Thank you to Dave Perry (Jennie Lee Research Labs); Joe Mills, Richard Hearne and the E&AVP department; Jim Hoyland (Music Studio); Peter Seabrook (Acoustics Research Group); and Álvaro Faria and Paul Garthwaite (Statistics Advisory Service); for all their technical help and advice. Thanks to Sidney Underwood for his craftsmanship and Gerard Roma for his hardware advice with the Woodentable. Several workshops have been very useful for this research. My big thanks to their organisers and participants: special thanks to Eva Hornecker, Loraine Clarke, Carey Jewitt, Sara Price, Kim Sandholdt, Anne Adams, Elizabeth FitzGerald, Katie Wilkie, Rose Johnson, Simon Holland, and Grégory Leplâtre. I am thankful to Stuart and Tracey Creal from the INTECH Science Centre for their help and support. v vi Thanks to all the PhD students for exciting chats and coffee breaks at the JLB student area, including in no particular order: Minh Tran, Aleksandra Pawlik, Lionel Mon- trieux, Pierre Akiki, Ann Abraham, Tamara Lopez, Laura Plonka, Alex Stiver, Jacky Bourgeois, Saad Saleem, Stefan Kreitmayer, Koula Charitonos, Robina Hetherington, Nadia Pantidi, Jennifer Ferreira, Brian Plüss, and Tu Anh Nguyen. My gratitude to the academics and fellows of the Music Technology Group in Barcelona. Thanks (in random order) to Martín Haro, Ferdinand Fuhrmann, Dmitry Bogdanov, Emilia Gómez, and Perfecto Herrera for offering me the opportunity to collaborate. Also thanks to Xavier Serra, Alba Rosado, Waldo Nogueira, Mattia Schi- rosa, Enric Guaus, Andres Lewin-Richter, Graham Coleman, Sebastián Mealla, Car- les F. Julià, Daniel Gallardo, Lilia Villafuerte, Sonia Espí, Cristina Garrido, Lydia Garcia, and everyone I forget here, for interesting conversations along the way. Many thanks to Colin Johnson and Luana Micallef for inviting me to give a seminar to the School of Computing, University of Kent. Special thanks to Carey Jewitt and Sara Price for a year full of intellectual engagement at the London Knowledge Lab. Also thanks in no order to Mona Sakr, Laura Benton, and Naomi Buneman for their enlightening advice. Last but not least, thanks to Anna Waring, Berit Henriksen, Ben Goodstein, Jenifer Ho, Kate Cowan, and all the IOE people for making my stay in London and at the LKL in particular a wonderful time. Thank you so much to colleagues and friends who kindly read draft chapters: Carey Jewitt, Gerard Roma, and Bill Adams. My gratitude to the Sutcliffe family, especially Panayiota, and to Jonathan Fine, for their hospitality during the beginning of this journey in the UK. Also many thanks to Lidia Daza, Natallia Yakavets, and Simona Nicoara, for their help and wise words. Since I finished my degree, it has been a long and precious journey. I feel fortunate to have shared it with family, friends, and colleagues: thanks to all of them, with special thanks to my parents Sebastian and Leonor, and my sister Marta, for their constant support and encouragement. I would like to thank the Open University in Milton Keynes (UK) for funding this research in the best environment one could imagine. Finally, my most important thank you to my friend, colleague, and partner Gerard. Preface The work presented in this thesis has led to the following publications, in chronological order: 1. Xambó, A., Laney, R., Dobbyn, C., & Jordà, S. (2011, July). Collaborative music interaction on tabletops: An HCI approach. In BCS HCI 2011 Workshop on When Words Fail: What Can Music Interaction tell us about HCI? Newcastle Upon Tyne, UK. 2. Xambó, A., Laney, R., Dobbyn, C., & Jordà, S. (2012, September). Towards a taxonomy for video analysis on collaborative musical tabletops. In BCS HCI 2012 Workshop on Video Analysis Techniques for HCI. Birmingham, UK. 3. Xambó, A., Laney, R., Dobbyn, C., & Jordà, S. (2013). Video analysis for evalu- ating music interaction: Musical tabletops. In S. Holland, K. Wilkie, P. Mulhol- land, & A. Seago (Eds.), Music and Human-Computer Interaction (pp. 241–258). London: Springer-Verlag. 4. Xambó, A., Hornecker, E., Marshall, P., Jordà, S., Dobbyn, C., & Laney, R. (2013). Let’s jam the Reactable: Peer learning during musical improvisation with a tabletop tangible interface. ACM Transactions on Computer-Human Inter- action, 20(6), 34. 5. Xambó, A., Roma, G., Laney, R., Dobbyn, C., & Jordà, S. (2014, June–July). SoundXY4: Supporting tabletop collaboration and awareness with ambisonics spatialisation. In B. Caramiaux, K. Tahiroglu, R. Fiebrink, & A. Tanaka (Eds.), NIME ’14: Proceedings of the 14th International Conference on New Interfaces for Musical Expression, 30 June – 4 July 2014, London (pp. 40–45).