tracking software for VST hosts

Chris Nash

Volume 1 – Report (including accompanying CD-Rom)

Submitted as part fulfilment for the degree of M.Phil.

Music & Media Technologies Dept. of Electronic & Electrical Engineering and School of Music, Trinity College Dublin

2004

declaration I hereby declare that this thesis has not been submitted as an exercise for a degree at this or any other University and that it is entirely my own work.

I agree that the Library may lend or copy this thesis upon request.

Signed: ______Date: ______

Chris Nash VSTrack: Tracking Software for VST Hosts Page 2

abstract

Tracking is a highly-developed method of music production that is divorced from the conventional MIDI sequencer and musical score approaches. Originating on the Amiga platform (with MOD files), tracker software traditionally comprises an internal sample-based synthesis engine and distinct “spreadsheet”-like user interface, which is sometimes likened to a digital pianola or step-sequencer – providing a powerful music platform without necessitating specialised hardware. Despite its relative merits, tracking is a technology that doesn't enjoy mainstream focus or credibility in the professional music world. With most composers firmly ensconced in sequencer packages, and given the general ‘technophobia’ of musicians, it seems the best way to draw them to this technology is to bring the technology to them – by importing the tracker interface and architecture into a sequencer. Taking the role of human-computer interaction (HCI) in music as a point of departure, this project reviews both the merits and the potential of such an endeavour, together with the pitfalls and limits of existing approaches. Notably, focus is given to tracking’s greater reliance on the computer keyboard – rather than the more ubiquitous mouse – as a solution to closing the interaction gap between man, machine and music. Previously, tracking software (the tracker) compromised on user-friendliness for the sake of power and efficiency – the mirror of sequencers. With VSTrack, we envisage several methods of redressing this balance, with minimal impact on the power, flexibility and ease-of-use (rather than ease-of-learning) of trackers, before proceeding to implement an initial prototype: VSTrack v1.0 – a tracker as a VST-plugin. Furthermore, a favourable and promising reaction to VSTrack is achieved by way of both an Internet release and live demonstration. Though further work is prompted, there is a clear consensus that a VSTrack plug-in, with the additions of minimal mouse and surround sound support, promises the best of both worlds, with the potential to solve many significant problems inherent in MIDI, sequencing and current tracking practices.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 3

abstract 3

chapter 1 introduction 6 1.1 about the report 9 1.2 about the CD 9

chapter 2 background 10 2.1 human computer interaction (HCI) and music 11 2.1.1 the fundamentals of interface design 12 2.1.2 computer interfaces 17 2.1.3 musical interfaces 29 2.2 MIDI (Musical Instrument Digital Interface) 36 2.2.1 introducing MIDI 37 2.2.2 disadvantages with MIDI 41 2.2.3 case study: the sampler as a MIDI device 43 2.3 tracking 46 2.3.1 introducing tracking 46 2.3.2 comparison with MIDI 53 2.3.3 case study: 2 57 2.3.4 case study: the tracker 65 2.4 sequencing 69 2.4.1 introducing sequencers 69 2.4.2 comparison with tracking 75 2.4.3 case study: Cubase 77

chapter 3 concept 80 3.1 application 82 3.1.1 benefits for tracking musicians 83 3.1.2 benefits for sequencing musicians 84 3.1.3 applications in surround sound 86 3.1.4 applications in video and film 88 3.1.5 applications in live performance 89 3.2 the scope of VSTrack 90 3.2.1 thesis objective 91 3.2.2 future versions 92

chapter 4 implementation 97 4.1 system architecture 98 4.1.1 the VST host system 99 (alternative platforms; programming language; initialising the VST architecture) 4.1.2 host / client interaction 104 (process communication and data flow; playback synchronisation; further integration) 4.1.3 audio processing 109 (sample synthesis engine; pattern interpretation and realisation) 4.1.4 storage format 114 (memory architecture; file format specification; backwards compatibility) 4.2 graphic user interface 122 4.2.1 the graphics environment 122 (VSTGUI (VST ); beyond VSTGUI: dynamic bit-map graphics) 4.2.2 the editor window 126 (the pattern editor; the sample list; the order list; other screens and dialogs) 4.2.3 input device handling 138 (keyboard support; mouse support) 4.2.4 integrated support systems 143 (online help, etc.)

chapter 5 evaluation 146 5.1 subjective response 146 5.2 objective response 148 5.2.1 internet testing and feedback 149 5.2.2 the VSTrack Advanced Preview seminar (TCD) 152

chapter 6 conclusion 157 6.1 future work 158 6.2 acknowledgements 159

bibliography 160

Note: Appendices A to D bound separately, in Volume 2.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 4

"In the studios that use advanced technology, we should look less toward inventing fresh sonorities and more toward defining and developing new conceptual organisms capable of generating new musical processes…" – Luciano Berio [14]

Chris Nash VSTrack: Tracking Software for VST Hosts Page 5

chapter 1 introduction Music, since the turn of the 20th Century, has been the story of composers trying to get closer to the fundamentals of sound. Composers, from Wagner through Berio and Stockhausen to Reich, have outgrown the confines of the traditional score, as technology has given them capabilities previously unheard of. Technological progress, however, stems not only from imagination, but also machination – the computer has become the new orchestra and with it comes the new score notation: MIDI. However, since its inception in the early 80's, MIDI has suffered much the same criticism as its forebear – the level of command necessary for the control and expression of today's music is still lacking. In the same decade, away from commercial and academic forces, an alternative method for computer music developed: tracking. Large – and invariably very young – communities of computer experts combined to create an

'underground' music scene, where members wrote, released and distributed not only their music, but the software and technology used to create it. Originating on the Amiga platform (with MOD files), tracker software traditionally comprises an internal sample-based synthesis engine and distinct “spreadsheet”-like user interface, which is sometimes likened to a digital pianola or step-sequencer – providing a powerful music platform without necessitating specialised hardware. The MIDI and tracking approaches to music both have their respective advantages and disadvantages, so it is in this project – with VSTrack – that we hope to harness the best of both worlds. In this section we will introduce the concept of VSTrack and its intended uses before familiarising the reader with the structure of this report.

concept Despite the relative merits of tracking (discussed in chapter two), it is a technology that does not enjoy mainstream focus or credibility in the professional music world. With most composers firmly ensconced in their sequencer packages, such as or Emagic Logic, and given the general ‘technophobia’ of musicians, it seems the best way for them to experience tracking, is to take it to them – by importing a tracker interface and architecture into a sequencer. This is the goal of this project – VSTrack: a tracker coded as a plug-in to a sequencer. VSTrack (pictured in fig.1.1) will use the VST (Virtual Studio Technology) open architecture for plug-in design, developed by Steinberg, to integrate with the sequencer application: receiving synchronisation information directly from the host, and returning multiple channels of high-quality 'tracked' audio. The net result is that the composer can retain the familiar sequenced tracks and "multi- tracking" audio utilities of the host, while simultaneously benefiting from the advantages offered by trackers.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 6 fig.1.1 – screenshot of VSTrack the application

Figure 1.2 shows a screenshot of the VSTrack application running in such a host application – Steinberg Cubase SX. Cubase is the industry standard music composition tool for the modern professional studio and its competences, as such, extend to recording, processing, arranging and even auralising both MIDI and waveform audio.

fig.1.2 – screenshot of VSTrack in Cubase SX

Chris Nash VSTrack: Tracking Software for VST Hosts Page 7 application In addition to offering a much finer command of sample playback and obvious potential in the realms of algorithmic programming, tracking is a completely different approach to writing music that encourages a new perspective from the composer. Conventional approaches make a compromise between the exploitation of the enormous power of and the retention of intuitive and familiar musical instrument interfaces. To this extent, MIDI has been a vessel to such a compromise, uniting these disparate worlds with debatable success [40][49]. The net result is that the musician can go on playing their piano, guitar or clarinet (in the form of their MIDI incarnates) as they have always done, with very little regard to the computer. Tracking advocates that the computer itself is an instrument with inordinate potential. However, it also acknowledges, as has been demonstrated time-after- time throughout computer history, that the best way to control the computer is with the standard 10x-key alphanumeric keyboard [1][12][26][44][46][68]. Thus, with tracking, it is the computer that becomes the extension of the musician, rather than a device connected via MIDI to the computer. Additionally, through the elimination of abstraction layers (such as MIDI), a tracker exploits the fact that it is not only the interface, but also the sound source – or, more precisely, it is an interface with intimate knowledge of the sound source, which MIDI lacks through its generality. In most cases, this sound source is a sample-based synthesis engine – itself a powerful component of most studios. However, the syntax for tracking (discussed in chapter two) allows more to be done with the samples on-the-fly, permitting a more expressive and, almost paradoxically, exact performance Practically, there are some areas of music to which tracking is well-suited, and some that aren't. Tracking, in the commercial Unterhaltungsmusik ('entertainment music') world, for example, is an especially effective way of programming percussion, as well as dub-based music. In the more academic Ernstemusik ('serious music') world, tracking has already found use in the areas of minimalist and algorithmic music. This project, however, is not only about introducing the advantages of tracking to sequencer users, but also to bring the many advantages of sequencer / multi- track hard-disk recorder packages to tracker users. Before continuing, though, it is necessary to familiarise the reader with the technology, terminology and general background of our endeavour.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 8 1.1 about the report

This report discusses the details and background of the VSTrack concept, proceeding to document the realisation of an initial implementation with basic functionality. Each chapter and section of the report is preceded by a brief description of the contents to follow. The overall layout, however, as implicit in the contents page, is as follows: In chapter 2, we begin by reviewing relevant literature and research available on the subject. Then, in chapter 3, once the reader has been suitably acclimatised to the domain of the project, we will be able to further expand on the concept and applications, previously introduced in section 1.0, and quantify the scope of our endeavour. This allows us to proceed to document both the design and implementation of a software prototype, in chapter 4. As is necessary for all software, a period of testing ensues the development, the fruits of which are reflected upon in chapter 5. Then, finally, we draw the project to conclusion in chapter 6 with a brief discussion of the priority of future work, previously introduced in chapter 3. Augmenting the report, in Volume 2, the reader should find several appendices, including the full of our implementation, in Appendix B. The code has been rigorously commented and indexed with references corresponding to explanatory paragraphs in the report. Such references are given at appropriate points throughout the text.

1.2 about the CD

Accompanying this report, the reader should find an enhanced CD-Rom. The disc is comprised of two components: an audio track featuring a demo song, created with VSTrack (and Steinberg Cubase SX) and playable in any audio CD player; and a computer data track, the content of which is as follows:

· /Report This report in soft-copy (Adobe Acrobat – .pdf) format.

· /Source The source code and MS Visual Studio .NET solutions (both VS .NET 2002 and VS .NET 2003 versions).

· /Binaries The VSTrack plug-in in pre-compiled, executable format (v0.84.2, including release and installation notes).

· /URL Links A selection of VSTrack-related Internet shortcuts.

· /Seminar The seminar presentation and handout (tutorial & survey) (to start presentation, press Ctrl-Enter after loading).

· /Proposal The presentation detailing the original proposal, presented in February, 2004 (to start presentation and turn pages, press Ctrl-Enter).

· /Demo The demo song in VSTrack module format (with samples & Cubase SX Project – .vstr must be loaded separately)

Chris Nash VSTrack: Tracking Software for VST Hosts Page 9