Master Software Engineering Master's Thesis Binary Differencing for Media Files Vladimir Komsiyski July 2013 Supervisors: Jurgen Vinju Jeroen van den Bos Abstract Modern digital asset management systems face the problem of handling hundreds of thou- sands of media files. These include multiple versions of the same image file or image editing software document, leading to substantial disk space requirements and long transfer times to and from the clients of the system. In an attempt to reduce the used storage and transmission times, we look into encoding only the differences between two versions of the same file and use them as a substitute for further processing. This work investigates several such approaches and evaluates their benefits and limitations. The variety of different file types and their nature makes it very difficult to identify the best performing method for all file types and further improvement or combination of approaches must be sought. Contents 1 Introduction 1 1.1 Problem Description and Motivation......................... 1 1.2 Outline ......................................... 2 1.3 Scope .......................................... 2 1.4 Related Work...................................... 3 2 Media Files 4 2.1 File Formats ...................................... 4 2.2 Data Compression ................................... 5 2.3 Image Modification................................... 6 3 Binary File Differencing Tools8 3.1 VCDiff and xdelta ................................... 8 3.2 bsdiff .......................................... 9 3.3 rdiff ........................................... 10 4 Approach 11 4.1 Differencing Tools with Media Files.......................... 11 4.2 Differencing Tools with Decompressed Media Files ................. 12 4.3 Video Compression................................... 13 4.4 Signatures and Fingerprints.............................. 14 4.5 Patch Generation.................................... 14 5 Evaluation 18 5.1 Test Environment.................................... 18 5.2 Binary Differencing Tools ............................... 19 5.3 Binary Differencing Tools with Decompressed Image Files............. 26 5.4 Video Frame Differencing on Image Files....................... 30 6 Conclusion 34 6.1 Summary ........................................ 34 6.2 Limitations ....................................... 35 6.3 Future Work ...................................... 35 A Test Data 37 v Chapter 1 Introduction 1.1 Problem Description and Motivation The quantity of existing media content grows with rate of GB/s. For example, hundreds of billions of photos are taken each year. In addition, the size of the newly created files is greatly increased by the quality of the capturing hardware. Recent cameras can produce High Definition video and images of size that was unthinkable a few years ago. While the average person can manage the private collection of pictures, movies and audio, the media industry face a new problem when dealing with the vast quantity of data. Digital asset management systems are used in the international media industry to manage large quantities of media content. Millions of photos, videos, articles, document and presentation files are used by newspaper and magazine publishers, production companies and marketing or sales departments. Publishing and media editing software is heavily used to modify such files to prepare them for release in a product. Graphics, audio and video editing naturally results in multiple versions of the same file. Traceability and back-up requirements force storing the old version in addition to the new one, which can result in hundreds of different versions of the same file, kept either on a server, or locally. Thus, managing such large quantities of media content is very storage expensive. In addition to the considerable amount of disk space requirements, the growing file sizes cause the transfer of a newly created file to the server to take a relatively long time. Retrieving an old version from the server also results in a long data transfer time when dealing with large files. For illustration, a typical publishing company has rates of 10,000 incoming images per hour on average. Most of them are standard JPEG files with sizes around 2 MB, which results in bandwidth of about 45 Mbit/s - this is 4 times the maximum bandwidth of the wireless standard 802.11b. Furthermore, peaks of up to 50,000 incoming images per hour can be observed, which increases the required maximum bandwidth to at least 200 Mbit/s for a single publishing company. In addition, the average rates impose storage requirements of several hundred GB added each day. With the continually growing bandwidth and storage capacity it seems to be a 1 2 CHAPTER 1. INTRODUCTION matter of time until the problem eventually solves itself, but the size media files and production material is just as likely to show similar growth in the years to come. In this work we will address these problems by investigating possible solutions for reducing the transfer time and storage requirements of versioned media files. 1.2 Outline Naturally, reducing the required disk space involves storing less files or minimizing the file sizes. As discussed, availability of all versions of a file is required and does not allow removing any of them. Reducing file size is done by various compression techniques that sometimes result in loss of data. Lossless compression, on the other hand, does not significantly reduce the file sizes. Different compression techniques are discussed in more detail in Chapter2. We will exploit the fact that the original file is accessible by both the server and the client. Since any editing takes place on the client, it needs to first retrieve the last (or any) version of the file in interest. The server stores all versions to date, so both the server and the client have access to the file prior to modification. This allows for the client to only compute a patch file - the difference between the original and the new version of a file. Transmitting only the changes made instead of the whole newly created version can dramatically decrease the file transfer time. The patch can then be applied to the original file on the server to recover and store the new version, or it can be stored as-is as a replacement of the file itself, and applied only when this specific version is requested by a client, thus efficiently storing multiple versions of the same file. In order for this approach to be useful, the time for creating a patch and transferring it to the server has to be less than the time it would take to send the whole file. Furthermore, receiving a patch and applying it has to take less time than simply receiving the new file. The current tools used for this kind of differencing are discussed in Chapter3. The fact that the nature of the media files and the most common modifications performed on them are known allows us to optimize the current tools in an attempt to achieve smaller patch sizes and faster run-time. Taking this to a next level: making the assumption that the server has already stored millions of files, similar data blocks might already exist on it. By using a fingerprint algorithm, it may be possible to locate any possible matches for the data blocks in the file being uploaded. This will allow even new files to be uploaded by just transferring a fingerprint and then only uploading data block, for which no matches were found on the server. The custom approaches are described in Chapter4, and evaluated along with the off-the- shelf tools with respect to saved disk space and transfer time reduction in Chapter5. Chapter6 summarizes the findings and concludes this work. 1.3 Scope Numerous media types and file formats exist. There are hundreds of different formats only for graphics files. Investigation of audio, video, vector graphics, documents, etc. is well beyond Binary Differencing for Media Files 1.4. RELATED WORK 3 the scope of this work and we will limit ourselves to raster graphics. Still, our approach can be extended for an arbitrary file type, not limited only to media. Since versioning of images comes naturally with editing, files used by graphics editing and publishing software will be investigated along with the most common graphics formats. Taking this into consideration, we try to answer the question of whether and when differenc- ing of media files is useful by first looking into the performance of current binary differencing tools on such files, how the images are typically edited, and what remains common among them. 1.4 Related Work The idea of using file differencing in order to reduce network traffic is relatively recent. The bsdiff and bspatch tools were created by Colin Percival in the early 2000s for FreeBSD security update compression. Currently they are successfully used by FreeBSD and OS X to transfer software updates to client machines, where the precomputed patches are applied and the new version of software is retrieved. Mozilla also makes use of these tools in their FireFox web browser updates [11, 15, 16]. Google took this approach one step further with Courgette, being developed to reduce the size of the Chromium web browser updates [17]. It uses knowledge of the specifics of the compiled application updates to optimize the performance of the binary differencing tool bsdiff , achieving 90% reduction of the size of the patches. The same idea is employed in the current work - performing binary differencing with a custom method extending an off-the-shelf tool, rather than directly with the tool. This allows to input knowledge of the target files to achieve better performance, which is essentially the same idea as in Courgette. Samteladze and Christensen conducted research with a similar objective [16]. Their aim was to reduce the network traffic for smart phone apps. They also used bsdiff and showed that downloading only the difference between the current and the new version of an app can reduce traffic by 48% on average. Their work, however, is focused on executable files. In all of the mentioned research a binary differencing tool is used to reduce the size of executable files similar to their previous versions and thus allowing for the generation of a patch.