Mt-Djing: Multitouch DJing Table Pedro Lopes Dissertation for the degree of Master in Information Systems and Computer Engineering J´uri Chairman: Professor Doutor Jos´eManuel Nunes Salvador Tribolet Adviser: Professor Doutor Jo~aoAnt´onioMadeiras Pereira Co-adviser: Professor Doutor Alfredo Manuel dos Santos Ferreira Junior Jury Member: Professor Doutor Nuno Manuel Robalo Correia October 2010 Acknowledgements As DJ history itself, this thesis has been a long journey, full of turning points and milestones. The astonishing number of people I would like to acknowledge shows how numerous were the dark alleys and obstacles I had to transpose during this last year. Firstly, to my father and mother for outstanding support, not only through this work but for the two decades before. To my advisor and co-advisor, Prof. Jo~aoMadeiras Pereira and Prof. Alfredo Ferreira Jr., for strong belief on Mt-Djing from square one. Moreover I kindly acknowledge the contribution of my colleague Diogo Mariano, whom I thank for all the support given, endless brainstorming sessions and late work nights. A word of appreciation to Ricardo Jota and Bruno de Ara´ujo,for the interest and support to this work, shown in the form of the many conversations exchanged upon this subject. Other colleagues have their place within this thank-you note, namely Tiago Ribeiro and Guilherme Fernandes, for their help in reviewing and discussing the contents. And at last, but (definitely) not least, to all DJs and accompanying group, that took the time and effort to contribute so much to this work - that ultimately is theirs more than it is mine - specially: Pedro Farinha, Pedro Sousa (aka Alfredo Carajillo), Jo~aoGomes (aka Mush Von Namek), Igor Sousa and Nuno Moita. This work was funded in part by the European Commission's Seventh Framework Programme (FP7/2007-2013) through project MAXIMUS, grant agreement IST-2007-1-217039 and through FCT MIVIS PTDC/EIA/104031/2008. Lisboa, October 2010 Pedro Lopes To the role of DJs in modern music. Towards becoming part of music itself. Resumo A hist´oriado DJing encontra-se repleta de inova¸c~oesao n´ıvel tecnol´ogico. Contudo, apesar destes dispositivos permitirem novas funcionalidades aos DJs, existe uma lacuna em produtos que ofere¸cam uma interac¸c~aot´actilsob uma aplica¸c~aode DJing Virtual, encontrando-se estas tipicamente reduzidas aos dispositivos de entrada tradicionais. Recentemente diversas propostas endere¸camt´opicosco-relacionados com a actividade do DJing; contudo, as met´aforasapresentadas n~aos~aocoerentes com o l´exicogestual do DJ, exigindo um extenso per´ıodo de treino. Atrav´esdo envolvimento de um grupo de DJs peritos, desenvolvemos uma solu¸c~aomulti-toque que possibilita o mapeamento directo dos conceitos fundamentais do modelo mental dos DJs numa aplica¸c~ao Virtual. Na solu¸c~aoproposta, a interac¸c~aogestual do setup Tradicional apresenta-se coerente com as met´aforasdesenhadas; e herdam-se as vantagens da virtualiza¸c~aodo dom´ıniodo DJing. O sistema aborda os principais requisitos do DJ contempor^aneo,como identificado na nossa investiga¸c~ao,salientando-se: robustez, baixa-lat^encia,controlo externo, adaptabilidade, suporte a plugins ´audioe assente em normas para a comunica¸c~aocom solu¸c~oesexistentes. A solu¸c~aoproposta permite ainda melhorias no desempenho das tarefas de DJing, contribuindo com uma interface totalmente reconfigur´avel em tempo real e adapt´avel `asnecessidades do utilizador. O sistema permite aos DJs exercerem a sua criatividade, criando cen´ariosde uma forma natural, que de outra maneira n~aoseriam poss´ıveis nos setups t´ıpicos.A compara¸c~aoda nossa proposta relativamente aos setups Virtuais, Tradicionais e H´ıbridascomprova a adequabilidade de uma superf´ıciemulti-toque para utilizadores experientes que desejem abandonar o setup Virtual em prol duma interac¸c~aomais natural. Abstract Disc-jockeys have come a long way, through technological evolutions. This path led them to the status and recognition they have achieved in our society. But as impressive as those technological evolutions are, as far as DJing is concerned, there are still few applications that support hands-on interaction over virtual DJ applications, and those are typically reduced to the traditional input devices. Furthermore, recent proposals apply new DJ metaphors, but are not successful in maintaining coherence with traditional DJ lexicon of gestures and concepts, thus requiring an extensive learning period. Accounting user feedback from an accompanying group of DJ experts, we devised a multitouch so- lution that maps physical core DJ concepts into a virtual application. In the proposed solution, gestural interaction from Traditional Setups remains coherent while inheriting advantages from the virtualization of the DJing domain. The system addresses typical requirements of contemporary DJing, as identi- fied by our research, namely: robustness, low-latency, external control, adaptability, audio plugins and connectivity standard-compliance. Additionally, we support task improvements, such as dynamic re-routing of music flow, seamlessly merging edit with DJ-performance mode and rearrangement of interface layout according to user needs. Our system allows DJs to exercise creativity with a natural interaction, creating scenarios that are not possible in the real world. Comparison against Traditional, Virtual and Hybrid DJ setups showed how a multitouch DJing surface developed with DJ involvement, can suit experienced users changing to Virtual setups. Palavras Chave Keywords Palavras Chave Interac¸c~aoMulti-toque DJing Concep¸c~aoCentrada no Utilizador Interface Natural de Utilizador Keywords Multitouch Interaction DJing User-centered Design Natural User Interface Index 1 Introduction 3 1.1 Context and Motivation . .3 1.2 Problem Statement . .4 1.3 Contributions of our work . .4 1.4 Publications . .4 1.5 Outline . .5 2 Background 7 2.1 Historical Developments on Djing . .7 2.2 Traditional DJ Setup . .9 2.2.1 Sound Sources: Turntable and CD players . 11 2.2.2 DJ mixer evolution . 11 2.2.3 Setup Layout . 12 2.3 DJ Performance Analysis . 12 2.3.1 DJ Research and Literature . 13 2.3.2 Accompanying DJ group . 13 2.4 DJ Requirements . 14 3 Related Work 17 3.1 Virtual DJ Setup . 17 3.1.1 Ableton Live . 18 3.1.2 Virtual DJing Applications . 19 3.1.3 Discussion on Virtual DJ setups . 21 i 3.1.4 Virtual towards Vynil Tracking . 21 3.2 Hybrid DJ Setup . 22 3.2.1 Discussion on Hybrid Setups . 26 3.3 Tangible Approach . 27 3.3.1 AudioPad . 27 3.3.2 BlockJam . 28 3.3.3 Reactable . 28 3.3.3.1 Scratch Implementation on the Reactable . 29 3.3.4 Pin & Play & Perform . 30 3.3.5 Discussion on Tangible approaches . 31 3.4 Wearable approach . 32 3.4.1 Sensor-Glove for Skipproof . 32 3.4.2 DJammer . 32 3.4.3 MusicGlove . 33 3.4.4 Discussion on Wearable approaches/devices . 33 3.5 Haptic approach . 34 3.5.1 D'Groove . 34 3.5.2 ColorDex . 34 3.5.3 Discussion on Haptic approaches . 35 3.6 Multitouch Controllers . 36 3.6.1 Lemur . 36 3.6.2 Attigo . 38 3.6.3 Stanton SCS.3d and 3m . 39 3.6.4 Multitrack scratch controller . 40 3.6.5 Discussion on Multitouch Controllers . 40 3.7 Setup Comparison and Conclusions . 41 ii 4 MtDjing 43 4.1 DJ Technical Requirements . 43 4.2 Mt-Djing Architecture . 44 4.2.1 Module Implementation . 46 4.3 Multi-touch DJ Interactions . 47 4.3.1 Understanding DJ gestures . 47 4.3.2 Outline of Interface Objects . 48 4.3.3 Generic Object Behaviour . 50 4.3.4 Object Connectivity . 51 4.3.5 DJ Object Handling . 54 4.3.6 Interface Canvas . 58 4.4 Extending Interactions . 60 4.4.1 External Control . 60 4.4.1.1 Footcontrollers . 60 4.4.1.2 Drumpads . 61 4.4.2 Integration with the professional DJ workbench . 62 4.5 Summary . 63 5 Evaluation 65 5.1 Test Environment . 65 5.1.1 Participants . 65 5.1.2 Apparatus . 66 5.2 Preliminary Evaluation . 66 5.2.1 Test Description . 66 5.2.2 Results and Discussion . 67 5.3 Final Evaluation . 68 5.3.1 Test Description . 69 iii 5.3.2 Results and Discussion . 70 5.4 Questionnaire and Open Speech Comments . ..