Click Tracker: Performance/composition tool for metrically complex scores
Joao˜ Pais∗ Pedro Lopes† Friedenstraße 14 Department of Information Systems and Computer Science 21098 Berlin INESC-ID/IST/Technical University of Lisbon Germany R. Alves Redol, 9, 1000-029 Lisboa, Portugal [email protected] [email protected]
Abstract duces this structure aurally. Many of the features Click Tracker is a tool designed for composers, conduc- necessary for the work in modern music are covered tors and instrumentalists working with modern music. and will be presented in the next sections. Further, It allows users to prepare an accompanying click track we will address technical aspects of Click Tracker’s of any musical score, independently of its complexity. features. Finally, we describe the numerous us- Also, several of its multimodal features take advantage ages of Click Tracker by performers and composers; of both visual and aural feedback; making it suitable for closing with our conclusions and depict upcoming musical study or learning contexts. The software is avail- features and improvements. able for public download and has been tested by several professional instrumentalists and composers, which re- sults are hereby discussed. 2 Click Tracker In this section we describe Click Tracker from Keywords two perspectives: user (who is interacting with it) Metronome, Composers assistance software, Music per- and technical aspects of the feature implementation. formance, Pure Data, Modern music Firstly we describe the dynamic metronome, that enables users to reproduce complex click accom- 1 Introduction panying tracks, following current trends on modern Click Tracker is a program based upon Pure Data[3] music. Followed by a description of user-generated designed for instrumentalists, conductors and com- content for the creation of the click tracks, the posers working with modern music. The main goal score syntax, and how to prepare language sound of the software is to prepare a click track of any files. Also from the user perspective the Graphi- score, no matter how complex it is. cal User Interface (GUI) is depicted and explained. Programmable metronomes and musical notation Finally Networking and Multi-user capabilities are software are nowadays widespread, allowing musi- discussed. cians to create basic training scores. But these tools 2.1 Complex Metronome do not account the needs of contemporary compo- sition, whereas the complexity of the score or the The Click Tracker feeds itself from a text file with learning context cannot be addressed. Our solution the specific syntax to generate a score (for detailed addresses these challenges and provides a simple example please refer to Appendix A). The com- tool, suitable for both realtime performance as well mands include: as training and composing situations. . beat durations: as integers, floats, or fractional Performers, conductors and composers working symbolic notation with new music often face challenges through- out score preparation of rhythmically demand- . regular bar structures: including untraditional ing pieces. Amongst these challenges we iden- bars and bars using irrational numbers tified: lack of aural feedback besides traditional . tempo changes: sudden or gradual metronome “clicks”, irregularity of beat durations, . repetitions changing bars, tempi variations, repetitions, and so . pick up bars forth. The Click Tracker reads the rhythmical structure . aural output, language and recording format defi- of a piece, written in a specific syntax, and repro- nitions
∗Click Tracker main developer and project leader. The aforementioned commands give the user a tool †HCI research, collaboration for GUI and this document. for manipulating durations, accommodating even the needs of unorthodox scores, as depicted in Fig- 2.2 Aural output ure 1. Click Tracker distinguishes itself from hard- ware/software metronomes or conventional scoring software because it also allows several aural capa- bilities apart from conventional clicks. Figure 3 de- picts the overall GUI of the current version, in the third section of the GUI (Figure 3.3) the user can choose how he wants the software to read the score: . conventional metronome click . voice (counting bar numbers) and click . only voice, counting beat numbers in the bar . voice, counting bar numbers plus beat numbers
Figure 1: Some duration features from the tutorial. Furthermore a wide range of languages is supported Numbers between [] indicate the original score syn- (English, German, Italian, Spanish and Portuguese). tax. Other languages are being prepared - as the produc- tion process is very simple, ideally all languages can The score is described as a list of individual be integrated into the software. events with specific durations: beats, or beat groups, in the case of regular bars. Modifier commands (such as tempo or other structural commands) pre- cede the beat(s) they affect. This information is in- terpreted by the patch, and organized into a group of arrays, which together generate a map of the score. These arrays, as depicted in Figure 2, store informa- tion regarding: beat duration, bar reference, tempi and bar database.
Figure 3: Click Tracker GUI components: 1) Per- formance display; 2) Performance control; 3) Out- put settings; 4) Transport and file control.
2.3 Score format The Click Tracker supports a wide number of score formatting options, amounting to complex options which our testers confirmed to be suitable for per- Figure 2: Internal arrays displaying the file score- forming contemporary music. These features are tutorial.txt, sent with the program. listed as follows:
For each event (each beat) of the score is created . Any duration for individual beats (integer of an index in the first three arrays. These are respon- float); sible for storing the required information regarding . Beat fractional notation in format x:y (e.g: 1:2 or current beat duration and tempo (after the tempo 3:7,8); changes have been pre-calculated). For external . Bar symbolic notation in format x/y (e.g.: 4/4 or reference, a fourth array contains the bar database, 3/9.17); which lists at which index each bar begins. To playback the score, the duration of each beat is . Tempo change at each individual beat; read sequentially, considering its original or scaled . Gradual tempo modifiers (accellerando / ritar- tempo. A delay object is used to time the durations. dando) in any amount of beats; . Repetition of any fragment (independent from the audio quality. Currently, the patch demands a tiny bar structure); amount of processing power. 2.5 Networking and Multiple-users Besides that, it is possible to bind the following performance parameters with each score: The Click Tracker supports networking via OSC protocol[5; 1], using it to communicate with the ex- . Number of pickup bars ternal GUIs, such as the one proposed in this pa- . Audio setting per. All the parameters and features displayed in the GUI are available as OSC tags. Therefore, it’s . Language a simple task to configure new control surfaces to . Recording format work bidirectionally with the Click Tracker. Currently, the OSC messages are divided in a Please refer to appendix A for a detailed example very simple way: of score format, further details regarding notation syntax are provided in[4], or in the tutorial score . GUI1 → Pd: all user-controlled commands, sec- packaged with the program. tions 2-4 of Figure 3; . Pd → GUI: realtime performance display ele- 2.4 Structure and preparation of language ments, section 1 of Figure 3; samples Each sample is provided as a wav file, stored in the Since all realtime informations of a performance folder with the sub-patches. To make the samples are available, it is a simple task to take any infor- as efficient and lightweight as possible (the bulk mation - such as the current score location - and use of the patch size is taken by the samples), only it to control any OSC-compatible patches (e.g. in whole numbers from 1 to 100, and then every hun- other software, such as Max/MSP, SuperCollider or dredth from 200 to 1000, are available. So, al- Csound). though the numbers 36, 136 and 236 aurally sound 2.5.1 Pd GUI as ”36”, after a first run-through it is quite clear The GUI in the original Pd patch is divided into four which number is meant to be playing - especially if main sections (Figure 3), according to task separa- the player is also looking at the GUI while rehears- tion: ing/performing. Besides the greater efficiency while preparing the audio files, it is also of advantage to · Performance display, where informations about have the numbers as short as possible. In some lan- the piece being played are displayed; guages, such as German, numbers with more than 2 · Performance control (velocity, volume, and so digits become very long words, which can’t be spo- forth); ken fast enough in a fast tempo. · Output settings (language, audio and recording The preparation and incorporation of new lan- formats); guages is a relatively simple process. A native language speaker with a clear diction - preferably · Transport and file control (play+stop, file opera- someone with some acting or speech-related back- tions); ground - is better suited to read the numbers in a Furthermore, it is possible to control the Click very precise, short, rhythmical, and yet understand- Tracker’s native GUI using keyboard shortcuts (re- able, way. After cleaning as much background noise fer to Appendix B) or turn the keyboard control off. as possible, the samples are synchronised aurally in 2.5.2 External GUI a DAW against a reference click track at constant intervals. The playback is made by playing a spe- Visualization and interaction with the click Tracker cific number (that is, excerpt) from a specific file, can be performed without recurring to the afore- directly from the disk. By not loading any sample mentioned Pd interface. In this case, an experimen- to the memory buffer makes it easier to spare re- tal interface was devised in Processing/Java, aiming sources, without interruption of any ongoing DSP for cross platform compliance and easier deploy- chain. ment amongst desktop and mobile scenarios. The For now, the samples are recorded in mono CD- external GUI takes advantage of Pd’s networking quality wav format. In the future, they will be down- capabilities to communicate through OSC messag- sampled to smaller sampling and bit rates - this will ing, as depicted in Figure 4. Any interaction on make the patch even lighter (and easier to use in mo- 1Or any other software/hardware controller, since the basis bile platforms), without any noticeable loss in the here is Client↔Server paradigm. the Interface will be communicated to the Pd patch, intensively used in all music-related contexts: for while any changes of the current performance state study, performance and composition of pieces. (e.g. tempo, beat or bar numbers) will be communi- The use of the program in varied real-life situa- cated to the Interface for displaying purposes. tions gave us a thorough insight about many small details to change, ranging from small user-friendly ClickTracker (PD) suggestions to bug reports. The usual feedback with state beat beat beat beat tempi musicians who used the software, as well as a small init stop play change questionnaire of the known users allow us to draw Experimental Interface (Java) out the following conclusions: Timeline started started · musicians use the Click Tracker mainly for con- temporary music, when the metrical or tempi Figure 4: Communication between click tracker and structure is very complex. Composers such as an instance of the experimental interface. B. Ferneyhough, B. Lang, E. Nunes, and E. Poppe (among others) fit better into this cat- The Interface is depicted in Figure 5, and follows egory; the same organization of the original GUI, however · in pieces of B. Ferneyhough with a simple bar it provides additional features, such as: structure (usually x/8 at slow tempi), but very · hiding of content (such as minimizing the ”Set- complex rhythms, usually with several tuplet tings” tab); layers, several musicians adopted the approach to prepare a metrical structure of strategic · drag and drop of score files to anywhere on the points - instead of the normal bar structure; interface, which will cause the new score to be loaded automatically; · it is used mostly at rehearsals, as musicians prefer to play without interference on stage - unless a · reset any parameter with a mouse right click (if a precise coordination e.g. with electronics pro- mouse is used as input device); cessing or tape is necessary; · operating remotely via OSC may facilitate in re- · independently of the complexity of the musical hearsal situation, where the Pd patch can run piece, or of the computer experience level of on a computer and the Interface on a tablet or the user, it is the general opinion that the cur- phone. rent structure allows musicians to program (al- most) all the elements necessary, while doing so in a very simple environment. In two par- ticular cases, it took a composer (with com- puter experience) around 10 minutes to input the structure of his 15 minute piece; for a pi- anist with very few computer experience, it took him several hours to input a piece of B. Ferneyhough with a very complex rhythmical structure, while at the same time recalculating all tempi to fit one of the available tuplet layers (around 2 or 3 at any given time) of his choos- ing; · the answer to the question ”did the program fulfil Figure 5: Experimental interface supports new fea- your needs and expectations” varied between ture, such as drag and drop scores. ”yes” to ”completely” (although, of course, still many improvements can be done); 3 In-performance Testing · users said they advised the use of the Click Tracker to other musicians. The Click Tracker has been available as a private project for a couple of years. The first public pre- Among others, the program is now used by sentation was made at the Darmstadt New Music the following ensembles: Nadar Ensemble (Bel- Courses in 2010. Since then, several instrumen- gium), Jack Quartet (USA); performers: Pavlos An- talists, ensembles and composers have joined the toniadis (Greece), Anna d’Errico (Italy), Katrien group of active users. The Click Tracker has been Gielens (Belgium) and Matteo Cesari (Italy); and the following composers: Aaron Cassidy (USA), 5 Acknowledgements Robert Dahm (Australia) and Mesias Maiguashca Our thanks go to the people who encouraged and/or (Ecuador/Germany). These are just the active users, helped the development of the program, either by as many other musicians have benefited from click trying it and reporting on it, and to the 1 composer, tracks made for the pieces performed. 1 performer and 1 ensemble who gave financial sup- Furthermore, although no users disagreed with ei- port for further development. ther features and/or usability, it was still sometimes hard to make traditional musicians (even when they References play untraditional music) to try out the software, [1] A. Chaudhary. Automated testing of open- or to integrate it into their workflow. This re- source music software with open sound world veals that more musician/composer-friendly fea- and open sound control. In ICMC 2005, tures, such as an appealing GUI, integration with 1/11/2005 2005. commercial notation software, midi import/export or the mobile/web platforms maybe be worth re- [2] PD. Pdanywhere. 2011. searching into. http://sourceforge.net/ projects/pd-anywhere/. 4 Conclusions and Future Work [3] M. Puckette. Pure data: another integrated com- Throughout this research we describe the major fea- puter music environment. In in Proceedings, In- tures of the proposed software, showing how it can ternational Computer Music Conference, pages assist users to create accompanying click tracks for 37–41, 1996. complex scores. The expanding list of musicians [4] C. T. recourses. that benefited from Click Tracker over the last years http://code.google.com/p/ enable us to conclude upon its adequacy towards clicktracker and https://www. various musical scenarios: composing, performing facebook.com/Click.Tracker, 2011. and learning. Latest research efforts have been made to create [5] M. Wright, A. Freed, and A. Momeni. Open external networked interfaces through the integra- sound control: State of the art 2003. In In- tion of the OSC protocol, allowing the software to ternational Conference on New Interfaces for be controlled through an external GUI and accom- Musical Expression, pages 153–159, Montreal, modate (and synchronize) multiple users. Further- 2003. OpenSound Control. more, the described GUI implementation (in Java) will enable it to be hosted entirely on mobile plat- 6 Appendixes forms, such as Android, as well as others were Pd is 6.1 A: Score Syntax available[2] - including a Pd Webserver. The following is an overview of the score syntax As for the future, many features are planned, ei- used to create click tracks. The text after the # char- ther just extensions of existing features - e.g., the acter is considered a comment and ignored by the curve design of tempo changes; the integration of parser. new languages (Norwegian and Flemish are already t <float> # tempo in MM, e.g., t 70 b # beat duration, a afterwards marks the beginning of a bar recorded); or new features - bar subdivisions, inte- ¯ gration of ad-hoc samples; richer score syntax, and : # fractional notation for duration, e.g., 2:3
Figure 6: Score of the provided Tutorial, exempli- fies supported features of click tracker.