Demo Session 2: Kids, Arts, Music, Theatre & Augmented Reality TEI 2018, March 18–21, 2018, Stockholm, Sweden Tquencer: A Tangible Musical Sequencer Using Overlays Martin Kaltenbrunner Jens Vetter University of Art and Design University of Art and Design Linz, Austria Linz, Austria [email protected] [email protected] ABSTRACT contemporary digital music practice. Compared to commonly This article discusses the design and implementation of the used software-based laptop instruments, tangible interfaces Tquencer, a tangible musical sequencer device. The tangible can provide a more expressive musical interaction bandwidth interaction paradigm for musical sequencers has been explored through direct physical manipulation to the performers, and previously trough the arrangement of simple tangible tokens therefore a more engaging performative experience to the au- within time-based physical constraints. While this initial ap- dience. Nevertheless we can also observe that apart from a proach allows for the creation of basic rhythmic or melodic few exceptions such as the table-based modular synthesizer loops, it sometimes lacks the musical depth required for pro- Reactable[7], most tangible musical interfaces often still lack fessional live performance. Therefore we introduce several the necessary musical depth for live performance, in spite of additional physical design elements that allow the develop- their innovative interaction design approach. Our Tquencer ment of musical complexity, while maintaining the overall design therefore intends to provide a portable and low-cost simplicity of tangible interaction. These tangible elements musical device, which combines several advanced tangible include single-step, multi-step and resizable tokens, which can interaction techniques in order to achieve sufficient musical be arranged to trigger musical events, multi-step sound effects complexity, while maintaining the overall simplicity of token- and more complex sequence patterns. Furthermore the device based interaction. also allows the dynamic assignment of the content and behav- Our tangible interaction design primarily includes the sequen- ior of physical tokens through dedicated configuration tokens. tial arrangement of physical tokens in combination with tan- We also introduce tangible overlays, which allow the handling gible overlays in order to expand the spatial limitations of a and arrangement of multiple musical configurations in various basic 8-step sequencer layout. We also focus on a fundamen- layers. This token configuration and layer management is tal tangible programming principle, which includes the static facilitated by an additional control step, which allows for the and dynamic assignment of musical content and functionality overall tangible configuration of a musical setup. Apart from to tokens and overlays within the principal tangible domain. its primary musical application scenario, the Tquencer also Therefore we provide single step tokens for musical events provides a generic tangible controller platform for time-based and resizable multi-step tokens e.g. for sound effects, as well sequencing applications. as dedicated configuration tokens for dynamic content assign- ment. For the placement of these configuration tokens as well Author Keywords as the tangible overlays we expanded the eight sequencer steps Musical Instrument; Tangible Interaction; Interaction Design; with an additional control position. Tangible Programming; Sequencer; RFID Sensing RELATED WORK INTRODUCTION There already exist several examples of tangible user inter- Token-based sequencing[10], which allows the arrangement faces, which individually implement some of the interaction of digital data embodied within physical tokens, is already design elements mentioned above. Verplank’s et.al. early a well established paradigm in tangible interaction design. Tile-Sorting Board[11] from 1994 has been later also adapted Sequencing is conceptually grounded in the musical domain to a musical application context. Bennet’s Beat Bearing[2] and has therefore been implemented in a vast variety of mu- device for example constitutes an elementary form of a tangi- sical instruments, starting with the musical automata in the ble sequencer, where the positioning of uniform metal tokens 18th century. Today this also includes several examples of within a physical multi-track configuration determines the tangible sequencer applications[1], since tangible interaction content and timing of musical events. Hesse’s Bubblegum promised to resolve some of the performance limitations of Sequencer[5] on the other hand introduces colored tokens with Permission to make digital or hard copies of all or part of this work for personal or statically linked musical content, which is independent of the classroom use is granted without fee provided that copies are not made or distributed vertical track position. Alonso’s Scoretable[6] implements a for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the circular token sequencer, which is a common alternative to author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or the standard linear topology. Newton Dunn’s BlockJam[8] republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. design on the other hand allows the programming and arrange- TEI ’18, March 18–21, 2018, Stockholm, Sweden ment of independent tangible sequencer blocks without the © 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. need for an external physical constraint. Costanza’s Audio ISBN 978-1-4503-5568-1/18/03. $15.00 D-Touch[3] allows the swift exchange of various sequencer DOI: https://doi.org/10.1145/3173225.3173233 429 Demo Session 2: Kids, Arts, Music, Theatre & Augmented Reality TEI 2018, March 18–21, 2018, Stockholm, Sweden topologies by using application sheets. A further non-musical also provides the necessary integrated sound output as well as example for the sequential tangible programming paradigm an onboard Ethernet port, which we use for overall configu- is Gallardo’s Turtan[4], which transfers the procedural Logo ration purposes and external control capabilities. Additional programming language into the tangible domain. Ullmer’s USB ports also allow the connection of arbitrary USB devices, Casiers[9] introduce the concept of tangible overlays, which such as external sound cards or MIDI converters, while the constitute context-dependent and exchangeable physical con- integrated Micro USB port is used for the power supply. The straints for interactive surfaces. This concept has been recently overall internal hardware components are shown in Figure 1. also adopted by the Sensel Morph1 device, which provides various controller overlays for its force-sensitive surface. HARD- AND SOFTWARE DESIGN The central interactive element of the Tquencer device con- sists of an array of eight+one RFID reader modules, which are each capable of detecting the simultaneous presence of two ISO14443 (Mifare) tags operating at 13,561 MHz. The individual RFID reader modules are based on the MFRC-522 chipset, already include a PCB antenna and are interconnected through a serial SPI bus configuration. In order to avoid cross- detection problems between nearby readers, we employ a time-multiplexed sensing method by sequentially activating and reading from each module individually. The actual number Figure 1. The internal hardware components of the Tquencer: of usable RFID tags is virtually unlimited, and we can stati- a) RFID array b) micro-controller c) single board computer cally (by design) and dynamically (by configuration) assign d) audio output e) touch controller f) controller knobs an individual tag UID to the actual token functionality. Hardware Components Software Components The overall interactive input and visual feedback is managed In conjunction with the basic I/O firmware on the micro- by an Arduino Micro (Leonardo compatible) micro-controller controller, we developed a simple background application, board, which includes an ATmega32U4 running at 16MHz. which constantly reads the provided interaction data from the We use almost all available I/O pins for connecting the various serial port and translates the according states and events to hardware elements. As mentioned above the RFID reader ar- a custom protocol based on Open Sound Control.[12] The ray is connected through the SPI bus and uses nine additional according OSC messages are primarily used for the control of digital selector pins. A complementary capacitive touch con- the internal synthesizer, and are simultaneously broadcasted troller based on the MPR121 chipset is connected via the I2C to any device connected via Ethernet for alternative external bus and provides twelve individual touch points. Furthermore control purposes. we use three analog pins for connecting the potentiometer The central synthesizer engine of the Tquencer has been imple- knobs required for the control of global parameters. The vi- 2 sual feedback is provided by an array of nine Neopixel RGB mented using the Supercollider sound programming language, LED modules, which are connected serially through single which not only provides the core musical sequencer function- digital pin. The whole device setup is powered by a 5V2A ality but also allows the design of simple programmable