tracking software for VST hosts Chris Nash Volume 1 – Report (including accompanying CD-Rom) Submitted as part fulfilment for the degree of M.Phil. Music & Media Technologies Dept. of Electronic & Electrical Engineering and School of Music, Trinity College Dublin 2004 declaration I hereby declare that this thesis has not been submitted as an exercise for a degree at this or any other University and that it is entirely my own work. I agree that the Library may lend or copy this thesis upon request. Signed: _____________________________ Date: ____________________ Chris Nash VSTrack: Tracking Software for VST Hosts Page 2 abstract Tracking is a highly-developed method of computer music production that is divorced from the conventional MIDI sequencer and musical score approaches. Originating on the Amiga platform (with MOD files), tracker software traditionally comprises an internal sample-based synthesis engine and distinct “spreadsheet”-like user interface, which is sometimes likened to a digital pianola or step-sequencer – providing a powerful music platform without necessitating specialised hardware. Despite its relative merits, tracking is a technology that doesn't enjoy mainstream focus or credibility in the professional music world. With most composers firmly ensconced in sequencer packages, and given the general ‘technophobia’ of musicians, it seems the best way to draw them to this technology is to bring the technology to them – by importing the tracker interface and architecture into a sequencer. Taking the role of human-computer interaction (HCI) in music as a point of departure, this project reviews both the merits and the potential of such an endeavour, together with the pitfalls and limits of existing approaches. Notably, focus is given to tracking’s greater reliance on the computer keyboard – rather than the more ubiquitous mouse – as a solution to closing the interaction gap between man, machine and music. Previously, tracking software (the tracker) compromised on user-friendliness for the sake of power and efficiency – the mirror of sequencers. With VSTrack, we envisage several methods of redressing this balance, with minimal impact on the power, flexibility and ease-of-use (rather than ease-of-learning) of trackers, before proceeding to implement an initial prototype: VSTrack v1.0 – a tracker as a VST-plugin. Furthermore, a favourable and promising reaction to VSTrack is achieved by way of both an Internet release and live demonstration. Though further work is prompted, there is a clear consensus that a VSTrack plug-in, with the additions of minimal mouse and surround sound support, promises the best of both worlds, with the potential to solve many significant problems inherent in MIDI, sequencing and current tracking practices. Chris Nash VSTrack: Tracking Software for VST Hosts Page 3 abstract 3 chapter 1 introduction 6 1.1 about the report 9 1.2 about the CD 9 chapter 2 background 10 2.1 human computer interaction (HCI) and music 11 2.1.1 the fundamentals of interface design 12 2.1.2 computer interfaces 17 2.1.3 musical interfaces 29 2.2 MIDI (Musical Instrument Digital Interface) 36 2.2.1 introducing MIDI 37 2.2.2 disadvantages with MIDI 41 2.2.3 case study: the sampler as a MIDI device 43 2.3 tracking 46 2.3.1 introducing tracking 46 2.3.2 comparison with MIDI 53 2.3.3 case study: Impulse Tracker 2 57 2.3.4 case study: the Renoise tracker 65 2.4 sequencing 69 2.4.1 introducing sequencers 69 2.4.2 comparison with tracking 75 2.4.3 case study: Steinberg Cubase 77 chapter 3 concept 80 3.1 application 82 3.1.1 benefits for tracking musicians 83 3.1.2 benefits for sequencing musicians 84 3.1.3 applications in surround sound 86 3.1.4 applications in video and film 88 3.1.5 applications in live performance 89 3.2 the scope of VSTrack 90 3.2.1 thesis objective 91 3.2.2 future versions 92 chapter 4 implementation 97 4.1 system architecture 98 4.1.1 the VST host system 99 (alternative platforms; programming language; initialising the VST architecture) 4.1.2 host / client interaction 104 (process communication and data flow; playback synchronisation; further integration) 4.1.3 audio processing 109 (sample synthesis engine; pattern interpretation and realisation) 4.1.4 storage format 114 (memory architecture; file format specification; backwards compatibility) 4.2 graphic user interface 122 4.2.1 the graphics environment 122 (VSTGUI (VST Graphical User Interface); beyond VSTGUI: dynamic bit-map graphics) 4.2.2 the editor window 126 (the pattern editor; the sample list; the order list; other screens and dialogs) 4.2.3 input device handling 138 (keyboard support; mouse support) 4.2.4 integrated support systems 143 (online help, etc.) chapter 5 evaluation 146 5.1 subjective response 146 5.2 objective response 148 5.2.1 internet testing and feedback 149 5.2.2 the VSTrack Advanced Preview seminar (TCD) 152 chapter 6 conclusion 157 6.1 future work 158 6.2 acknowledgements 159 bibliography 160 Note: Appendices A to D bound separately, in Volume 2. Chris Nash VSTrack: Tracking Software for VST Hosts Page 4 "In the studios that use advanced technology, we should look less toward inventing fresh sonorities and more toward defining and developing new conceptual organisms capable of generating new musical processes…" – Luciano Berio [14] Chris Nash VSTrack: Tracking Software for VST Hosts Page 5 chapter 1 introduction Music, since the turn of the 20th Century, has been the story of composers trying to get closer to the fundamentals of sound. Composers, from Wagner through Berio and Stockhausen to Reich, have outgrown the confines of the traditional score, as technology has given them capabilities previously unheard of. Technological progress, however, stems not only from imagination, but also machination – the computer has become the new orchestra and with it comes the new score notation: MIDI. However, since its inception in the early 80's, MIDI has suffered much the same criticism as its forebear – the level of command necessary for the control and expression of today's music is still lacking. In the same decade, away from commercial and academic forces, an alternative method for computer music developed: tracking. Large – and invariably very young – communities of computer experts combined to create an 'underground' music scene, where members wrote, released and distributed not only their music, but the software and technology used to create it. Originating on the Amiga platform (with MOD files), tracker software traditionally comprises an internal sample-based synthesis engine and distinct “spreadsheet”-like user interface, which is sometimes likened to a digital pianola or step-sequencer – providing a powerful music platform without necessitating specialised hardware. The MIDI and tracking approaches to music both have their respective advantages and disadvantages, so it is in this project – with VSTrack – that we hope to harness the best of both worlds. In this section we will introduce the concept of VSTrack and its intended uses before familiarising the reader with the structure of this report. concept Despite the relative merits of tracking (discussed in chapter two), it is a technology that does not enjoy mainstream focus or credibility in the professional music world. With most composers firmly ensconced in their sequencer packages, such as Steinberg Cubase or Emagic Logic, and given the general ‘technophobia’ of musicians, it seems the best way for them to experience tracking, is to take it to them – by importing a tracker interface and architecture into a sequencer. This is the goal of this project – VSTrack: a tracker coded as a plug-in to a sequencer. VSTrack (pictured in fig.1.1) will use the VST (Virtual Studio Technology) open architecture for plug-in design, developed by Steinberg, to integrate with the sequencer application: receiving synchronisation information directly from the host, and returning multiple channels of high-quality 'tracked' audio. The net result is that the composer can retain the familiar sequenced tracks and "multi- tracking" audio utilities of the host, while simultaneously benefiting from the advantages offered by trackers. Chris Nash VSTrack: Tracking Software for VST Hosts Page 6 fig.1.1 – screenshot of VSTrack the application Figure 1.2 shows a screenshot of the VSTrack application running in such a host application – Steinberg Cubase SX. Cubase is the industry standard music composition tool for the modern professional studio and its competences, as such, extend to recording, processing, arranging and even auralising both MIDI and waveform audio. fig.1.2 – screenshot of VSTrack in Cubase SX Chris Nash VSTrack: Tracking Software for VST Hosts Page 7 application In addition to offering a much finer command of sample playback and obvious potential in the realms of algorithmic programming, tracking is a completely different approach to writing music that encourages a new perspective from the composer. Conventional approaches make a compromise between the exploitation of the enormous power of computers and the retention of intuitive and familiar musical instrument interfaces. To this extent, MIDI has been a vessel to such a compromise, uniting these disparate worlds with debatable success [40][49]. The net result is that the musician can go on playing their piano, guitar or clarinet (in the form of their MIDI incarnates) as they have always done, with very little regard to the computer. Tracking advocates that the computer itself is an instrument with inordinate potential. However, it also acknowledges, as has been demonstrated time-after- time throughout computer history, that the best way to control the computer is with the standard 10x-key alphanumeric keyboard [1][12][26][44][46][68]. Thus, with tracking, it is the computer that becomes the extension of the musician, rather than a device connected via MIDI to the computer.