chapter 3 concept

"To add new editing functionality or a new type of display, for example, it should be easy to incorporate that new module into the system without the need to recompile the system." – R.S. Rubinstein [52] (on the requirements of modern sequencers)

In the last chapter, as well as introducing their underlying technologies, we established the principal advantages and disadvantages of both tracking and sequencing. In our comparisons, it has been shown that often the advantages of one approach are mirrored in the disadvantages of the other. For example, an advantage of sequencing is the resolution afforded by MIDI, whilst its lack thereof constitutes a significant shortfall in tracking. This prompts an important observation: trackers and sequencers can be seen as two pieces of the same jigsaw puzzle. The trouble is that developers, until recently, have taken to making music software with four straight edges. Current trends, however, are taking the client / server analogy of networking and distributed work into the realm of the single application, albeit under the guise of either child / parent or plug-in / host abstractions – endeavours that often 33 explicitly refer to their own jigsaw analogies . VSTrack, In VSTrack, we wish to construct a jigsaw with sequencer and tracker pieces. a VST plug-in To this end, we require a sequencer piece with at least one ‘open’ edge – a sequencer that is a plug-in host – into which we can slot (viz. plug) our own tracker piece. As mentioned in the last section, VST hosts [60] such as Steinberg Cubase SX meet this criteria, and thus form the basis for our project; VSTrack will be a tracker that operates as a VST plug-in. We shall explore the finer details of the chosen architecture and its design implications, in section 4.1.1. Modern sequencers, as discussed, already present more than one piece of the jigsaw – in addition to one MIDI sequencer piece they have integrated one multi- tracking audio recorder piece (see fig.3.1). If we look at the relationship between these two competences in the application as a whole, we see that they are in many ways mutually exclusive. For example, the score editor, piano roll, event list, etc. all stand apart from any audio editing. Indeed, the only real integration of the two comes in the form of the arrange window. This makes sense, since the arrange window, in controlling the song, must have control over all aspects in the work, including both audio and MIDI sequencing. The distinction is then made,

33 A commonly recognised use of the jigsaw analogy is that in the former Microsoft Office logo, which represents each Office component (Word, Excel, Access and Powerpoint) as a different coloured piece of a four-part jigsaw. Indeed, a similar and possibly more apt use of jigsaw imagery was also employed for previous versions of Microsoft’s Visual Studio integrated developer environment – that we use in our implementation.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 80 within the arrange window, by giving each competence a distinct and discrete track – audio tracks and MIDI tracks. The net result is that, aside from in the arrange window, our hybrid is largely uncompromising.

fig.3.1 – VSTrack as a VST plug-in34 S n i o - f g t - u l S p

a VST

m b r p

Host e l v e e r R

VSTrack

VSTrack identifies itself to the host as a VST Instrument (or VSTi), which essentially functions similarly to a MIDI track – albeit one where the logical (MIDI) output is sent to an internal plug-in, rather than an external hardware synthesiser. Consequently, an audio input is then fed back to the host from the respective plug-in or synthesiser, and manifests itself as a normal audio track. It is important to note that the principle input to VSTrack will not be this normal MIDI input from the host. Instead, it is the plug-in itself that defines how VSTrack’s internal sampler should perform, through the user’s manipulation of the plug-in’s editor window (i.e. the tracker’s own pattern editor); VSTrack ‘sequences’ itself. By this method, we are able to exploit the plug-in’s computer keyboard support in order to use the computer itself to provide the musical input. VSTrack therefore has command of the complete musical process, yielding audio entirely determined by its own user interaction and its own algorithms. Thus, like the audio / MIDI hybrid, VSTrack exists apart, without direct interference from the sequencer’s MIDI and audio capabilities. As such, we are not significantly constrained in our implementation and can almost unreservedly inherit all the features of trackers that give them their respective advantages, whilst retaining those already present in the sequencer host.35

34 It is worth noting that this schematic also suggests the possibility of connecting two VST hosts together – a possibility enabled by Steinberg’s VST System Link architecture. 35 As we shall see, in Chapter 4, there are some limitations with the VST architecture that do force us to subtract from the average tracker’s full feature set.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 81 about this chapter In the following sections, we will first explore the potential advantages and even solutions offered by a tracker plug-in such as VSTrack, before outlining the scope of our own endeavour. Trackers have the ability to cover all stages in music realisation, from inspiration to audio CD-ready realisation; the potential scope for a tracker plug-in is thus extensive, and requires us to carefully define our goals in this initial offering. However, aside from these practical considerations, we also take the opportunity to explore the theoretically practical – such as developments, improvements and features that will not be implemented in this project, but are nonetheless possibilities afforded us in the future36.

3.1 application

There are many benefits to this hypothetical hybrid and, in many ways, “the whole is greater than the sum of its parts” – the musician is afforded opportunities that they would otherwise have foregone, had they been restricted to a single approach. In the next few sections, we expound upon these rewards by focusing on the individual communities and aspects of musical production that would be effected. bringing together However, let us first make one observation that affects all communities and technologies and communities areas. As was made clear in the previous chapter, the groups and individuals involved in tracking and sequencing, respectively, are from differing backgrounds. We have charted the tracker community’s history, and noted its close links and common heritage with computer programmers (in some cases, ‘hackers’) – perhaps likening the use of early trackers to the ‘programming’ of music. Contrastingly, we have explored the sequencer community’s history, and noted its comparatively closer ties to not only musicians, but also the music industry at large. It is thus fair to conclude that the successful union of trackers and sequencers might not only offer more technical opportunities to each party (as we discuss in the coming sections), but also social and synergetic opportunities. Former tracking musicians will be encouraged to learn sequencing methods, but will subsequently approach them from a more technical standpoint than the average sequencer user, or musician. Former sequencer musicians will also have the opportunity to learn tracking methods and, in so doing, may instead engender a more artistic approach than the average tracker user. Of course, the final corner of the trinity, is the situation where the tracking artist and the sequencing artist work together as a team. In this latter case, each party’s respective technical and musical competences would potentially benefit their counterpart’s command of even their own area.

36 It is intended that VSTrack’s development will continue beyond the realms of this report and thesis.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 82 It is important to note, however, that the congruity between tracking and technical proficiency may derive from the complexity and technical demands of the traditional tracker interface. In this endeavour, it is hoped that our exploration of human-computer interaction (HCI), in Chapter 2, will help us to allay such problems. A successful hybrid will require its tracker interface to be not only easy- to-use, but also easy-to-learn.

3.1.1 benefits for tracking musicians

Although the VSTrack concept promises tracking musicians full access to the host sequencer, it is important to note that it has not been impossible (nor unprecedented) for musicians to combine the benefits of tracking and sequencing in the past. The process involved rendering the final output of the tracker to an audio file, then importing the audio file as an audio track into the sequencer. In a hybrid such as VSTrack, the tracker exists as an audio track. There is no rendering stage necessary – nor any cumbersome audio file importing. Simply, the VST host is played back and, at any stage, VSTrack supplies its own output in synchrony – for example, play the host from the third beat of the eighth bar and VSTrack will work out that it must start its playback at the twelfth row of the third pattern. Indeed, VSTrack is able to draw upon the facilities of the VST host architecture to not only synchronise on the level of musical or MIDI entities (beats, bars, measures, etc.), but also on the level of the sample; VSTrack is sample accurate. In typical scenarios, this means that VSTrack is accurate to at least one 44,100th of a second – even if the musician can’t yet track at this resolution. VSTrack’s representation as an audio track, like all VSTi’s, has further advantages. Notably, the output can be despatched to other effects – such as other VST plug-ins (VST Effects, as opposed to VST Instruments), or even via a soundcard output to external hardware effect processors. Though this was technically possible with the former render / import method, imagine a situation where you apply an effect in the host and find that something needs a little ‘tweaking’ in the tracker’s output – for example, where the host’s delay effect is producing a texture that is too dense, and you wish to prune the notes in the original feed. The rigmarole of returning to the separate tracker, re-rendering and re-importing the audio file would be enough for most musicians to give up on the entire endeavour. However, with VSTrack, one merely needs to open the editor window and make the changes directly. an example scenario In summary, let us take another hypothetical scenario: A composer wishes to write a piece of orchestral music with a strong rhythmical emphasis, as well as an element of musique concrète – in the form of a background environmental sound that lasts for the duration (e.g. traffic sounds, forest ambience, factory

Chris Nash VSTrack: Tracking Software for VST Hosts Page 83 noises, etc.). In such a situation, they can: import the background sound as an audio file to an audio track; record the instrumental and orchestral sounds (including any soloists) using a MIDI track; and draw upon VSTrack to engineer and programme the rhythmic and percussive elements. Respectively, they have: omitted the deficiency of trackers and long samples; omitted the deficiency of trackers and performance capture, due to inferior resolution; and harnessed two significant advantages of trackers – its competences in both rhythmic engineering and sample control.

3.1.2 benefits for sequencing musicians

The example in the last paragraph already highlights two of the advantages that a tracker addition to sequencers might have for sequencing musicians. However, whereas it may be true to say that most tracking musicians will have at least rudimentary experience with sequencers, the reverse is rarely true. As such VSTrack not only makes tracking more accessible to the sequencing musician, but may even introduce them to a new method of writing music that they had been previously unaware of. This ‘new method’ not only proffers the keyboard, over the mouse, but encourages a completely different and original perspective on music composition. In the same way that the strict and formal quantisation of pitch, inherent in the advent of tuning scales, enabled a new (and popular) perspective for many composers, trackers’ advocacy of a strict and formal quantisation of rhythm offers the same. Put simply: the reduced resolution of trackers forces the composer to think differently – not to compromise, but to innovate. In all music technology, electronic musicians must not limit themselves to simply using technology to realise their musical ideas, but also use it to inspire ideas in their own right – allowing it to perhaps suggest directions that had previously not occurred to them. tracking: a new Though it is beyond the scope of this thesis to suggest such new directions in perspective music, we are able to look back over recent developments in music and observe how trackers can be used as an effective tool in their pursuit. For example, the music of twentieth-century school of minimalism “depends for its effect on extended reiterations of the same short melodic motif, moving gradually through changes of key and colouration, imparting a haunting mantra-like quality to [its] scores” [19]. minimalist tracking The notion of a sampler (detailed in section 2.2.3) is already beneficial in this pursuit – for example, the short melodic motif can be represented by a single sample. Additionally, the use of the computer clipboard (cut-and-paste) analogy allows composers to write their motif, then repeatedly copy it, each time making slight modifications to harmony, melody or rhythm, etc.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 84 Trackers absorb both these capabilities, but also provide an alternative interface by which to affect them. Take, for example, Steve Reich’s use of phasing in music – the performance (or playback) of two lines of music at marginally differing speeds, so that new and original musical patterns are elicited when the lines fall out of synchronisation. Sequencers provide a useful method of finely adjusting the speed, to produce this effect gradually, as Reich originally advocated [47]. However, many would argue that the effect mostly draws upon the moments of the output where the rhythms of the two lines “lock” (as Irish composer, Donnacha Dennehy, terms it). This occurs, for example, when the temporal offset between the two lines is equal to some integral division of the bar (e.g. a sixteenth, an eighth, or so on). At such points, the notes “fuse” together and are perceived as one line of music [6]. Returning to the tracker interface, we see that all this requires is the duplication of a track / channel (using the clipboard) and use of the Ins (insert) key on the keyboard, to slide it along in steps of one sixteenth. serialist tracking Minimalism is often seen as a reaction to an earlier school of twentieth-century music: serialism. However, it is also easy to see applications for trackers in serialism. Originally, the “digital” (viz. discrete) nature of tuning systems led to the use of serialism in determining pitch sequences – earning serialist music the label of “twelve-tone music”. However, many composers, including Karlheinz Stockhausen and Edgar Varèse, began to apply these ‘sequencing’ systems to other variables, such as spatial distribution and dynamics. Like MIDI and most digital music systems, numbers control every aspect of the sound. Unlike MIDI sequencers, however, the tracker interface makes no attempt to hide these numbers for the users – perhaps by replacing them with ambiguous graphical representations – and thus the serialist tracking composer can simply calculate their sequence and then type it into whichever parameter they so desire – pitch, volume, panning, etc. tracking and More generally, trackers have the potential to address the tuning limitations of new tonalities MIDI, as mentioned in section 2.2.2. The effect column, of the tracker row provides us with the potential to implement a “detune” effect, whereby there exists a command for transposition up or down, and a parameter that decrees the magnitude in cents. Indeed, the existing commands EFx, EEx, FFx and FEx (see Appendix A) already provide similar functionality – though these are defined as pitch slides, the slide is applied before the note is sounded. However, the x parameter does not relate to pitch in terms of cents37. One need no longer resort to the relative nature of the MIDI pitch bend to affect microtonal tuning.

37 Note, however, that the Renoise parameter, E5x (also listed in Appendix A) does specify cent usage.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 85 the keyboard: an alternative The MIDI keyboard is so prevalent in the use of sequencers that it has become input device all but a requirement of their effective use – few have managed to solely rely on the mouse. Consequently, the hardware requirements list of sequencer software should not only reflect processing power, memory and supported operating systems (OS’), but also the necessity of a MIDI keyboard – and, more importantly, the musical proficiency that the MIDI keyboard requires. Trackers, when compared to sequencers, have the advantage that neither a keyboard nor significant musical experience is required [48]. Thus, it is also not inconceivable that the VSTrack / Cubase SX lay-musician might forego the MIDI component of the sequencer completely and just use VSTrack in combination with the audio facilities of the host.

3.1.3 applications in surround sound

We have already talked about the potential of tracking, in relation to overcoming the limitations of MIDI in the area of surround sound. Let us now take a moment to think about our plug-in from base principles, and consider how the audio should be returned from VSTrack, to the host. feeding audio to Both sequencers and trackers each have their own set of tracks, but the crucial the VST host observation is that we have described VSTrack as a single sequencer track. Instead, we could imagine a scenario where the VSTrack plug-in pipes 32 (or 64) mono audio tracks back to the host – one for each tracker channel. This way VSTrack doesn’t have to concern itself with the mixing together of the tracks, and can instead relegate this task to the host – where these facilities, such as mixers, EQ, effects, etc., already exist in abundance. However, though this approach clearly presents advantages in mono and perhaps even some stereo mixing, we are returned to the problem of addressing higher orders of spatialisation – once again the panning control is biased to the channel and not the note, and requires an excursion from the tracker to the sequencer’s mixer or arrange window. There thus remain two methods to regain the flexibility of panning control in the tracker notation: firstly, to pass the channels pre-panned – instead of sending 32 mono channels, send 32 stereo, quad or 5.1 channels; secondly, to continue to handle all the mixing in the tracker and pass a single 5.1 channel to the host. As the first would clearly overwhelm the host (memory- and DSP-wise), we are left with the second38.

38 A third option is also conceivable: VSTrack could provide a compromise number of channels. For example, 32 5.1 channels could be mixed internally into 4 or 8 5.1 busses, which are then sent to the sequencer for the master mix. Such a compromise represents a fair balance of flexibility. Indeed, this is a method / compromise similar to that in most hardware mixers, except that this author knows of few examples of mixers with surround sound bus outputs – a none within the budget of the average individual.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 86 fig.3.2 – the methods to spatialisation in MIDI (a) MIDI Devices Note Channel Spatial Position (with and without a sampler) and trackers Note

(b) MIDI and Note Channel Program* Spatial Position AKAI Sampler*

Note Spatial Position

(c) Trackers Note Channel Spatial Position

Note Channel Spatial Position

(* the AKAI program is able to redistribute the notes, by using their pitch to set specific spatial positions)

the sample-oriented Taking a different tack entirely, we could follow the example of the AKAI S6000 approach sampler (mentioned in section 2.2.3), which was able to assign individual samples to individual outputs. Like tracking, this avoids the channel-centric problem, and since each note specifies its own instrument, it could be likened to a note-centric approach. One might have 6 copies of the same sample, but each assigned to the different channels of a 5.1 layout – one need merely specify the appropriate sample to address the appropriate point source in the surround field. However, the AKAI had no dedicated surround sound support, and thus divided its inputs either individually or in stereo pairs, which makes dynamic two-dimensional panning very difficult. Fig.3.2 illustrates the various possible methods to spatialisation available to the electronic musician. tracking in There is not an extensive history of surround sound support in trackers. Impulse surround sound Tracker 2 had an option to encode a channel as the virtual rear speaker of Dolby

Surround (S91, see Appendix A). Modplug Tracker [29] developed on this with support for quad channels (whilst still using IT2’s file format), and allowing the internal reverb to be sent to the rear speakers, with additional panning control in the rear pair – but no panning from front to back. Renoise has support for an arbitrary amount of ASIO audio outputs, to which the user must assign a stereo pair to each tracker channel – again inhibiting front / back panning. As such, there is no tracker that directly and fully supports quad or 5.1 surround sound - or even identifies surround sound as a two dimensional quantity. an example Such failings in sequencing are due to the restrictions of MIDI, and fortunately implementation there exists no such limitations in tracker notation. One possible implementation would be: to augment the current stereo channel pan effect (Xxx – see Appendix A) with a channel depth pan effect (e.g. Yxx, which is currently assigned to Panbrello; or Zxx, which is currently assigned to user-defined macros39); similarly extend IT2’s S91 command to S9x, to control depth; and add a third mode to the

39 Indeed, the most common use of Zxx, was to take advantage of IT’s limited low pass filter, which could be easily migrated elsewhere.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 87 volume / panning column switch (detailed in section 4.2.3), to make it volume / panning (x-plane) / panning (y-plane). Of course, a more ideal surround system (i.e. 128 or more speakers in a circular arrangement [51] – possibly up to 800, in the case of Iannis Xenakis [49]) might simply use Xxx’s single parameter and, in combination with volume, exploit polar coordinates – as opposed to the other Cartesian option. Furthermore, 5.1 extensions, such as the centre and LFE (low frequency encoding) channels could be respectively absorbed into the 2D plane and given an individual alphanumeric identifier, such as “LFE” (addressed by the L key – which would identify it apart from any decimal or hexadecimal numbering system). Further benefits would come from surround sound-oriented processing, such as keyboard shortcuts that can flip channels, mirror coordinates, or calculate two-dimensional pans. Indeed, this latter option is all but already present – since it simply involves the interpolation of the beginning and end values in the x- and y-plane panning columns – a feature already available (albeit currently for the one plane). Were such functionality added to trackers, it would be of considerable advantage to not only tracking musicians, but also sequencing musicians – since our example composer at the end of section 3.1.1 could now effortlessly distribute his rhythm in surround sound. Indeed, not having to undergo the rigmarole of distributing MIDI objects in surround sound through the sequencer, would encourage many a composer to overlook the resolution difficulties of trackers and use VSTrack for more melodic and harmonic competences, as well as rhythm.

3.1.4 applications in video and film

An advantage to tracking musicians, not mentioned in 3.1.1, was the inheritance of the sequencer’s competences in video. For a long time, sequencers have given musicians the ability to synchronise their playback to video, using time codes such as SMPTE [65]. This is so that video, TV and film composers can preview the synchrony of the video’s visuals and audio with their own audio (the music). synchronising More recently, sequencers such as Steinberg Cubase SX have allowed the with video user to import a variety of formats of digital video into the sequencer itself – and have the video playback in its own window, adjacent to the arrange window, mixer, score, etc. Naturally, since we envision our tracker plug-in to be synchronised with sample-accuracy to the host, we can assume that the plug-in is also synchronised to any playing video. However, this perhaps small detail was not mentioned in section 3.1.1, because it is not the only consideration for which video is involved. VSTrack: a video Though VSTrack is designed to complement a sequencer, such as Cubase, it editor plug-in? has all the components necessary for a self-contained piece of music creation software (like other trackers). Its only requirement is a host that can interpret its

Chris Nash VSTrack: Tracking Software for VST Hosts Page 88 basic graphics and audio requests, so that they can be realised by the operating system (OS). Thus, not only would it be possible to build a simple VST Host, to make VSTrack a standalone application, but it would also be possible to add VSTrack’s competences to non-musical applications – such as video applications40. Thus, someone editing the parts of their video could – for example, in a hypothetical Adobe Premiere VST – compose or sketch the music for their feature, as they worked on the video. We likened music software, at the beginning of this chapter, to a jigsaw puzzle; an important observation is that the puzzle is not necessarily limited to audio in scope, but audio-visual. Current software allows us to plug the video-output piece into the music puzzle (as in the last paragraph), and also plug audio-output pieces into the video puzzle, but we should be able to plug the sequencer, tracker, audio editing and video editing pieces into the same puzzle. Possibly, with the acquisition of Steinberg, by video software giants, Pinnacle Systems, this might one day be a reality. video tracking Indeed, in such an application, another more original adaptation of tracking technology would be to take video as an input for the tracker (samples become video clips) and use the pattern editor to process aspects such as colour and playback speed, with the effect column servicing video effects.

3.1.5 applications in live performance

Berio [14] observes, in relation to electronic music, “We lost music in not only its technical, historical, and expressive aspects, but also its immediacy and its social aspects. We realized, for example, that an audience assembled to listen to loudspeakers is not particularly exciting.” Audiences like to at least have a sense of what goes into a performance. In acoustic music, the audience can see the connection between the movement of hands, fingers or a bow and the sound – even if they don’t understand which (or how) pitches are being excited. Thus, if we translate Berio’s loudspeaker example to the modern electro-acoustic performance, we have the same problem with the computer as live instrument. visual stimulus in Promoting the computer or laptop’s monitor to a screen that is visible to the the electro-acoustic performance audience can alleviate the tedium, but applications such as Max/MSP are often beyond the average viewer’s comprehension – and can be highly abstracted from the actual audible happenings. Tracking, on the other hand, offers an intuitive display by comparison, where – like a pianola – the fundamentals of the program are easily elicited by simply observing the program and its operation in action. As mentioned, though the tracker is not ideal for real-time performance, real- time composition nevertheless has its applications in entertainment. For example, a single pattern could be set to loop continuously, then built up progressively. As

40 It should perhaps be noted that this is not something the tracker architecture offers per se, but something offered by the necessity of making our VST Instrument able to encapsulate its own music (as opposed to simply responding to MIDI input). Indeed, a plug-in that might be more popular to existing film composers, might be a sequencer as a plug-in. This might also be useful for combining with standalone trackers that have VST hosting support – the reverse of VSTrack.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 89 the playback cursor runs through the pattern, the user can edit it – so that when the cursor comes round for subsequent iterations, the changes are realised. This incremental update technique seemingly lends itself to genres such as minimalism (detailed in 3.1.2), where gradual change is one of the principal methods and goals [47]. Indeed, nearly all genres have some form of progressive or modular aspect to them, be it respectively the gradual addition of complexity to a rhythmic or percussive part, or the chorus / verse structure of the song itself – both of which find parallels in the architecture of trackers.

3.2 the scope of VSTrack

For this project, though it would be nice to produce a cutting edge, market-ready surround sound-capable version of VSTrack, such a goal is overly ambitious. However, we are not obligated to produce such an extravagant offering in order to justify our claims – which can be succinctly summarised, as follows:

i. to demonstrate the large and relatively untapped potential of the computer keyboard in the field of computer music – notably, in competition with the computer mouse.

ii. to demonstrate the opportunities afforded by tracking software, an oft keyboard-biased method of computer music.

iii. to explore, more generally, the importance of the interface, in promoting the computer as an ‘instrumental’ musical device, rather than as an ancillary tool.

In the last section, we explored our hybrid concept’s potential with intentionally little consideration of practical constraints – including those of time, technology, music, finance, etc. In the next section, we qualify the actual and realistic scope of the implementation conducted adjacent to this report. Then, in the final section of this chapter, we slacken solely the constraint of time, and detail potential interface extensions to VSTrack that will be possible in the time immediately following completion of the thesis. To this end, we dub our implementation: version 1.0 – the first generation in an extended lineage.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 90 3.2.1 thesis objective

This thesis documents the methods and concepts involved in the production of a mature alpha version of VSTrack v1.0. To put this in perspective, the program’s

roadmap was initially defined as follows:

fig.3.3 – VSTrack thesis roadmap · v0.4 - Core system Completion of memory architectures (data types, fundamental architecture, etc.) and basic GUI (graphic user interface) systems.

· v0.8 – Initial Alpha version. Most (c. 90%) v1.0 features implemented, with minimal internal testing.

· v0.9 – Initial Beta version. All v1.0 features implemented, with nominal internal and targeted third party testing.

· v1.0 – Release version. All v1.0 features implemented, with rigorous internal and public testing. Testing subsequently becomes the responsibility of the user base (possibly prompting further versions – 1.01, 1.02, etc.).

Based on the above schematic, subtasks are then identified, weighted (according to time requirements) and scheduled. For example, the coding of the loading and saving algorithms might constitute a 0.1 increment in the current version designation. Naturally, earlier developments might have consequences for those tasks scheduled for later – leading to minor deviations in the designations of more major checkpoints. As such, the final version numbers for

this endeavour were as follows: core system, 0.37; initial alpha version (a1), 0.76;

second alpha version (a2), 0.8; third alpha version (a3), 0.84 – with the third alpha constituting our objective: a mature (viz. relatively stable and feature endowed) prototype. All relevant compromises are detailed in Chapter 4. the design goals VSTrack, whose interface is a development of the Impulse Tracker 2 (IT2) of VSTrack v1.0 interface (detailed in section 2.3.3), aims to inherit all the interface and audio capabilities of its ancestor. However, IT2 itself develops on the earlier Scream Tracker 3 – and it is with respect to both that we define Version 1.0. Version 1.0, in comparison to v2.0 or v3.0, is thus loosely defined: ST3’s sample-centric voice architecture; IT2’s extended effect list (defined in Appendix A); IT2’s user interface; and the superior audio quality of whatever VST host it is running in (i.e. Cubase SX). Naturally, any aspects of these previous programs that are seen as detrimental will be omitted – and, if necessary, replaced. One important omission in the version encapsulated by this thesis will be mouse support. Although steps will be taken to facilitate VSTrack’s later extension in this area, we aim to demonstrate that the mouse is peripheral to the

Chris Nash VSTrack: Tracking Software for VST Hosts Page 91 usage of the program. In section 5.2.2, this report documents a seminar where musicians are exposed to VSTrack for the purposes of feedback. In this seminar, we desire to force the attendees to confront the keyboard, in lieu of the mouse. We nonetheless concede, even at this stage, that mouse support would be instrumental in making the average user’s initial steps easier, making VSTrack easier-to-learn.

3.2.2 future versions

We foresee support for IT2’s remaining competences, such as instrument-centric voice architecture, MIDI support and possibly even its network functionality41, as a defining feature of VSTrack v2.0. The IT2 instrument model, as discussed in section 2.3.3 and similar to that discussed for MIDI samplers in section 2.2.3, is a marked improvement in sonic flexibility and expressivity, but nonetheless superfluous to our principally interface-oriented objective; trackers are still able to demonstrate their potential without such support. The musings of section 3.1, including surround sound, also constitute possible (and probable) developments for v2.0 and future major versions. Indeed, due to the large scope of VSTrack in the music production process, almost any cutting-edge audio or DSP (digital signal processing) technology might find application in the plug-in – for example: the use of FFT (fast fourier transform) to enable changes in the harmonic aspect of a sample (e.g. pitch), without incurring changes in the temporal aspect (e.g. rhythm, or even decay)42. VSTrack as VSTrack’s internal sampler, with the addition of MIDI connectivity, could software sampler function as a MIDI (VSTi) software sampler – with the MIDI messages feeding the tracker’s sampler, and bypassing its patterns. Together with the IT2 instrument model, VSTrack could be a comparable alternative to existing software samplers, such as Steinberg HALiON, Kontakt, or Tascam GigaStudio. Indeed, there is no reason why the sample and instrument (patch) file formats of these samplers – as well as hardware samplers, such as the AKAI S6000 (discussed in section 2.2.3) – can’t be adapted and loaded into VSTrack, with a suitable import algorithm. complete tracker- The logical next stage of MIDI / VSTrack integration is the ability to represent sequencer integration the sequencer’s MIDI tracks as tracker channels – allowing the user to switch interface for the same line of music. Though this adds some flexibility for the user in their approach to music, as regards the interface, the encapsulated music would still be restricted by the limitations of not only MIDI, when it came to output, but also trackers. For example, when someone writes a line of music in MIDI, then switches to the tracker interface, it would have to undergo quantisation, so

41 IT2’s limited audio data bandwidth (due to the use of lower quality samples) made networking practical in former years; it remains to be seen whether the increased requirements of today’s audio quality standard make such an endeavour practical even over modern network and Internet connections. 42 The Roland Variphrase sampler is a tangible example of this real-time approach to temporal-accurate pitch-shifting – albeit only an instance of the methods practicality in hardware.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 92 as to be accommodated by the row structure of tracker patterns. Likewise, when someone writes the music in trackers, then switches to one of the MIDI displays, they might lose significant panning effects or sample manipulations that are unique to tracking. In any case, however, the subject is a moot point – as, at the current time of writing and for the foreseeable future, the VST architecture hasn’t the required feature allowing plug-ins access to the contents of the host’s MIDI tracks. A compromise might be to implement a score notation display mode in the tracker itself – something the tracker notation lends itself to, as established in section 2.3.2. improving the Short of ‘defining-features’, there are a number of smaller innovations that could interface be introduced at almost any stage – perhaps for v1.0, even if not v0.84. Some that would affect the interface are discussed (and illustrated) in the remainder of this chapter. Graphical User Interfaces (GUI’s) are built on the bitmap principle; the transformations afforded a bitmap often have application in the interface, at a higher level. WIMP interfaces (see section 2.1.2), for example, exploit the ease by which it is possible to move portions of bitmaps around the screen quickly and efficiently, even allowing them to be covered and later uncovered. Another common operation for bitmaps is scaling – which, in the UI, can relate to ‘zoom’. pattern zoom There are three potential uses of ‘zooming’ for VSTrack. The first is a standard zoom out facility (fig.3.4). At any point in time, VSTrack v1.0’s pattern editor will be capable of displaying 5 tracks or channels (columns) and 32 rows – a limitation inherited from IT2 [67]. However, the size of a pattern is likely to extend to 64 tracks and 64 rows, perhaps beyond. A zoom-out facility, in VSTrack, therefore might show a macroscopic view of the pattern – though too small to affect any editing, detailed enough for the user to observe the overall layout of data in the pattern as a whole. In addition, mouse support could allow the user to drag the current ‘viewing window’ (the area normally displayed in the pattern editor) to another location in the pattern, for editing or closer scrutiny.

fig.3.4 – conceptual image of pattern zoom (out) feature

Chris Nash VSTrack: Tracking Software for VST Hosts Page 93 fig.3.5 – conceptual image of full screen mode

full screen or For editing purposes, a greater scope in the viewing window could of course be expert modes achieved through its direct enlargement. Fig.3.5 illustrates how the pattern editor could be displayed to maximise screen ‘real-estate’, relegating the other views (such as the sample list – as pictured) to floating windows. Similar views in other applications, such as Discreet 3D Studio Max, are known as expert modes; they hide any toolbars, menus or buttons and depend solely on the user’s knowledge of mouse and keyboard control. The second use of ‘zooming’ contrasts the first: a standard zoom-in facility. In some cases – such as the program’s use by the visually impaired, or the observation of the program from a distance (e.g. at live functions, on a projected screen) – the font size might be too small to be legible. A simple, but effective, method of addressing this is just to have each pixel in the program represented by a square of four pixels on the output screen – a doubling of height and width. Non-integer scaling (e.g. a 50% increase) would be less trivial, but nonetheless practicable. A better solution, in such circumstances, might even be to switch the pattern’s display of characters from bitmap fonts to (fixed-width) truetype fonts, such as those used in most word processors. pattern data zoom The third use is a logical extension of the second: a zoom-in facility, but one that zooms the pattern data. For example, instead of causing the characters to become larger on the screen, this type of zoom would allow the user to see more rows themselves – yielding a higher temporal resolution. VSTrack, currently, boasts 4 tracker rows to the musical beat. Now imagine that one is allowed to zoom the pattern, at iterative magnifications of x2 – the first zoom would allow a resolution of 8 tracker rows per beat; the second, 16; the third, 32; and so on, exponentially. The zooming, however, should perhaps not affect the entire pattern; this would prompt wide areas of unused rows and make the music harder

Chris Nash VSTrack: Tracking Software for VST Hosts Page 94 to view as a whole – as well as place exponentially more load on the processor. Instead, one could implement the feature, so that the user can select a specific block of rows (perhaps 16 rows – a musical bar in 4/4) and zoom in solely on this area, for areas where finer resolution is need. The concept is illustrated by the

conceptual image, in fig.3.6.

fig.3.6 – conceptual image of pattern zoom (in) feature

waveform preview Roads [49], as we observed in section 2.2.2, criticised the lack of cohesion between MIDI and timbre. This is analogous to the differences between music (in the abstract, as well as in common music notation) and sound – a chasm suffered by all notational forms. Tracking brings composers closer to the sound source, and gives them more command of it, but how can we represent this in the interface and the notation? Fig.3.7 demonstrates one method that might be used – simply displaying the waveform engendered by a particular note sequence (e.g. a tracker channel) aside or behind the note sequence itself. Since the graphic need only change when the notes are edited (and, even then, only at that position), there need be negligible impact on the program performance itself; neither pattern navigation or audio playback will affect the graphic nor involve any need to update the waveform display. Indeed, it could even be a feature that could be enabled or disabled for each individual channel. Such an addition would give the composer an overview of the actual sound of the performance, rather than the abstract music. More practically, the ability to see the waveform would benefit those working with, and trying to synchronise, rhythmic samples (e.g. loops, etc.). To draw the graphics representing the playback at all points in the track, the program must have knowledge of sample amplitude, etc. at midpoints in the sound. The cost of the calculation necessary to elicit enough information for graphical purposes is significantly less than that required for audio purposes (reduced enough to make it practicable), but nonetheless harkens back to the problem of beginning note playback from a point after its pattern entry (trigger) – the problem of long background sounds, highlighted in section 3.1.1. We observed that the audio facility of most modern sequencers represented a solution to this. However, there is nothing to prevent a similar audio facility in a tracker.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 95 fig.3.7 – conceptual image of waveform preview feature

fig.3.8 – conceptual image of audio track feature

audio tracks More specifically, a feature could be introduced that allowed the user to identify a specific audio file to playback, in the background, that would always start at the same point (perhaps the outset, or a user-defined offset). From this point, the tracker’s mixing engine need only involve itself in simple arithmetic to work out where to start the audio file’s playback. In VSTrack, we must already do similar calculations to provide synchronisation with the VST host. Thus, the clock signal that is received from the host could, in our plug-in, directly translate to the offset of playback in the tracker. Visually, this feature could manifest itself as a sort of ‘static’ audio track (as opposed to the current, more ‘dynamic’, tracks in VSTrack), and is illustrated as such, in the conceptual image of fig.3.843. In the next chapter, we set forth in documenting our implementation, but will

again touch on potential directions for the future in both chapters 5 and 6.

43 This may be seen, of course, as a step in the direction of assuming multi-track audio recording competences in the tracker environment – similar to those in modern sequencers. Though this might be inane in VSTrack, where such functionality can already be done in the host sequencer, it has been discussed as an option for other standalone trackers, such as Renoise.

Chris Nash VSTrack: Tracking Software for VST Hosts Page 96