Masaryk University Faculty of Informatics Linear Audio Editor Bachelor’s Thesis Josef Hornych Brno, Fall 2015 Replace this page with a copy of the official signed thesis assignment and the copy of the Statement of an Author. Declaration Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Josef Hornych Advisor: RNDr. Miloš Liška, Ph.D. i Acknowledgement First and foremost, my sincere thanks go to RNDr. Miloš Liška, Ph.D. for supervising me and helping me with my thesis. I would also like to thank the people who helped me test my application, namely Pavel Šindelka and Filip Holec. Without their input, several major errors in my application would go unnoticed. And lastly, my gratitude goes to the hosting team at Impact Hub Brno and my colleagues at Mathesio s. r. o. for their words of encour- agement and support. ii Abstract The goal of this thesis is to create a basic audio editor. The user interface of the implemented editor is focused around the use on tablet computers running Windows 8.1. Firstly, the term ’audio editing’ is defined along with the possible use cases for such editors on portable devices. The main part of the text contains the description of the implemented audio editor along with the the supporting libraries that were necessary to be created. The output of this thesis is a tablet-oriented audio editor, which supports manipulating multiple tracks, mono and stereo audio clips as well as three basic audio filters: normalization, fade in and fade out. It contains a software mixer through which a project created in the editor can be mixed down and exported to an output file. Along with the editor itself, this thesis helped spawn several code libraries useful for third-party developers, available under the MIT license. iii Keywords audio editor, tablet, Windows 8.1, zvukový editor, Windows Store iv Contents 1 Introduction ............................ 1 2 Audio Editing ........................... 3 3 Tools Used ............................. 5 3.1 NAudio ............................ 5 3.2 TypeScript ........................... 6 3.3 RequireJS ........................... 7 3.4 Knockout ........................... 7 4 Implementation .......................... 8 4.1 EditoneLib ........................... 8 4.1.1 Hierarchy . 8 4.1.2 Mixer . 9 4.1.3 Serialization . 10 4.1.4 Audio Format . 10 4.2 EditoneLibRT ......................... 11 4.2.1 Delegation . 11 4.2.2 Optimization . 13 4.2.3 Input and Output . 14 4.2.4 Bridge to APIs . 14 4.3 EditoneApp .......................... 15 4.3.1 Delegation . 15 4.3.2 LESS . 15 4.3.3 Editor . 16 4.3.4 Tiled Clip Rendering . 16 4.3.5 Open and Save Project Dialogs . 16 5 User Interface ........................... 18 5.1 Adherence to Design Guidelines . 18 5.1.1 Left Margins . 18 5.1.2 Flyouts . 19 5.1.3 Panning . 20 5.2 Examples of Tablet-Oriented Approach . 20 5.2.1 Clip Drag and Drop . 20 5.2.2 The Knob Control . 20 5.2.3 The AppBar . 21 5.2.4 Responsivity . 21 6 Platform’s Limitations ...................... 23 v 6.1 Debugging C# and JavaScript Simultaneously . 23 6.2 Custom Zooming and Native Panning . 23 6.3 Plug-ins ............................ 24 6.4 Folder Access ......................... 24 6.5 Discussion .......................... 25 7 Conclusion ............................. 26 7.1 Monetization Potential .................... 27 7.2 Final Words .......................... 27 Bibliography . 29 Index . 32 A Appendix: Archive Contents . 32 vi 1 Introduction Whether one is a musician recording their own song, an aspiring writer seeking to record their own audiobook or a professional audio engineer, a good audio editor is a necessity. There are many existing paid and free solutions for all of the well-established platforms, but some platforms lack the diversity. When I searched on the Windows Store[1] for adequate solutions for cutting and mixing multiple audio tracks, I was met with no results. This lead me to the idea to create an application which would fulfill the needs of an audio editing enthusiast on the go. Having previous experiences developing programs for audio manipulation, I found this to be an interesting challenge and a great choice of a topic for my bachelor’s thesis. The goal of my thesis is to create a linear audio editing application with user interface well suited for Windows tablet computers. Its design should follow the best practices for tablet-oriented applications as described in Microsoft’s design guidelines[2]. In the first part of my thesis, I describe what lies under the umbrella term of audio editing. I explain where my application fits in and what use cases it satisfies. The following two chapters are a discussion of how I implemented my solution. A list of the tools, libraries and frameworks I used is fol- lowed by the description of the components and the models I created for my application. Afterwards, I write about the user interface and I demonstrate how my application is optimized for use on tablets. I show a few examples of the GUI decisions I made and explain what makes them suited for touchscreen interfaces. The following chapter of my thesis contains the discussion of the problems and drawbacks I ran into. Some of these are intentional restrictions of the Windows Runtime platform, others are unexpected hurdles I had to jump or otherwise cope with. In each case I investigate the problem and arrive to a solution or a possible workaround. In the final chapter I summarize the work I have done for this thesis. I take a brief look on how my application could be monetized and I 1 1. Introduction conclude my thesis with thoughts on how I could further improve on my work in the future. 2 2 Audio Editing Editing audio on computers is ubiquitous and has become the stan- dard way of treating sound ever since computers became fast enough to handle it[3]. Computers, similarly to the revolution they helped to start in the video industry, completely changed what was possible with sound. The advent of freeware and open source programs such as Au- dacity meant that anybody could afford to do professional-sounding edits to their recordings. The ability to edit sound on computers is nowadays taken for granted. Audio editing, in its broadest sense, is any manipulation of sound, be it cropping, shifting, changing the volume, applying a compressor effect or equalization. On portable devices, some of these operations can be too demanding on the CPU and the battery. Fortunately, this does not concern the casual user, because the functionality they are using is mostly undemanding and easy to process. Take, for example, a journalist going by public transport back from an interview with a recording. They may want to cut some parts out, change the volume, mark up several spots, but they have no need to use an equalizer or the time to tinker with it. These sorts of operations are simple and can be performed in real time even on a portable device such as a tablet or a smartphone. The basic audio editing functions an editor should have include: ∙ Cropping, splitting and positioning (changing the offset of) au- dio clips ∙ Adjusting the volume of individual tracks ∙ Changing the panning of the clips ∙ Fade in and fade out effects ∙ Superimposing (mixing) multiple tracks at the same offset ∙ Mixing down multiple tracks to a single-file output There are already, of course, applications that offer just that. For the major platforms, like Android and iOS, reasonable options exist, some of which are even for free[4][5]. Many of them, however, lack the 3 2. Audio Editing ability to edit multiple tracks, or only support saving to a downmixed file as opposed to also offering to save the project as a work inprogress for later further editing. When Microsoft released Windows 8 and introduced Windows Store applications, I saw an opportunity to create the first audio editing application for the new platform. I set out to create an app which would feel native and be easy to use on a Windows tablet. I decided to name it Editone, a portmanteau of edit and tone. For Editone, I set these goals: 1. Create a platform-independent core library that could be used by third parties. The idea behind this library is to enable de- velopers to easily organize audio clips on a timeline, serialize and deserialize this data. It should provide an API for the basic audio editing functions I outlined earlier. 2. Write a simple audio mixer that supports mixing multiple au- dio tracks at once. This would allow the users to play their projects back immediately and export them to a down-mixed audio file. 3. Implement a set of basic audio filters that a user would need the most. The thesis’ topic mandates a fade-in and a fade-out effect and one other effect of my choice. 4. Create a user interface for use on tablets and other devices with touchscreens that would feel fluent, as uncluttered as possible, easy to use and, at the same time, familiar to a user experienced with other audio editing software. 4 3 Tools Used Editone is written using Visual Studio 2013[6]. Its front end is written in HTML5, while its back end is implemented in C#. In the following sections I describe the libraries and tools I used to help me develop Editone. 3.1 NAudio The NAudio library[7] is an open source project which aims to provide sound manipulation functionality for the .NET platform.