LoopMaker: Automatic Creation of Music Loops from Pre-recorded Music Zhengshan Shi Gautham J. Mysore CCRMA, Stanford University Adobe Research Stanford, CA, USA San Francisco, CA, USA [email protected] [email protected] ABSTRACT and games often use music as a backing track [6]. Such editors Music loops are seamlessly repeatable segments of music are often novices with respect to music and audio editing. One that can be used for music composition as well as backing of the reasons that a loop is well suited for background music tracks for media such as videos, webpages, and games. They is that it can be played for an arbitrary length of time and are regularly used by both professional musicians as well as sound continuous. This is particularly useful for webpages novices with very little experience in audio editing and music and games since the length of time that a user spends on a composition. The process of creating music loops can be given section is not pre-determined. It is also helpful for de- challenging and tedious, particularly for novices. We present ployment, as a loop can often be as short as a few seconds and LoopMaker, an interactive system that assists users in creating requires little storage. and exploring music loops from pre-recorded music. Our To create music loops, one can either compose them, or create system can be used in a semi-automatic mode in which it them from pre-recorded music. The focus of this paper is the refines a user’s rough selection of a loop. It can also be used in latter. A typical workflow to create such loops [1, 11, 22] a fully automatic mode in which it creates a number of loops using audio editing software is as follows: from a given piece of music and interactively allows the user to explore these loops. Our user study suggests that our system 1. Listen to a significant portion of a song or a whole song. makes the loop creation process significantly faster, easier, and more enjoyable than manual creation for both novices and 2. Roughly identify a part that would be a candidate for a experts. It also suggests that the quality of these loops are music loop. comparable to manually created loops by experts. 3. Find precise beginning and end points of the loop such that there is no abrupt sounding transition when played ACM Classification Keywords repeatedly. H.5.5. Sound and music computing: Methodologies and tech- niques: I.5.5.Implementation:Interactive systems 4. Use low level audio tools to crop this selection, creating the loop. Author Keywords Our user studies suggest that this loop creation process can Music loop creation; Interactive audio editing; Audio be challenging and time-consuming, especially for novices. applications. We present LoopMaker, an interactive system to assist users INTRODUCTION in creating loops from pre-recorded music. The goal is to A music loop is a segment of music that can be played repeat- automate the tedious aspects of this task (which our user study edly such that the transition from the end of the segment back suggests in step 3 and 4 above), allowing users to be involved to the beginning of the segment sounds natural and seamless. in the more creative aspects of the task (step 1 and in some Continuously playing a loop essentially sounds like a contin- instances, step 2). Based on this, LoopMaker has the following uous piece of music [18]. Loops are regularly used by both two modes of operation: professionals and novices (people with little experience in mu- Semi-automatic — The user listens to a song and finds the sic or audio editing) for a variety of applications. Musicians • rough locations of a desired loop (steps 1 and 2 in the work use loops to compose themed layers in their musical pieces flow). Our system then performs steps 3 and 4 to refine and and DJs often use loops to customize pre-existing music for create the loop. their specific needs. Editors of media such as video, webpages, Automatic — The user simply provides a song and the • Permission to make digital or hard copies of all or part of this work for personal or desired loop length. Our system automatically performs classroom use is granted without fee provided that copies are not made or distributed all four steps and creates multiple loops that fit the length for profit or commercial advantage and that copies bear this notice and the full citation criterion. Our system provides an interface for the user to on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, explore the created loops and select the ones that are most to post on servers or to redistribute to lists, requires prior specific permission and/or a suitable for the desired task. fee. Request permissions from [email protected]. CHI 2018, April 21–26, 2018, Montreal, QC, Canada The processing time in both modes is on the order of seconds. © 2018 ACM. ISBN 978-1-4503-5620-6/18/04. $15.00 We performed a user study with both novices and experts to DOI: https://doi.org/10.1145/3173574.3174028 Figure 1. Interface for LoopMaker. It can automatically create loops using the Find me Loops button. It can assist users in creating loops in semi- automatic mode using the Refine button after users roughly select a region using the cursor. determine the usefulness of our system by comparing loops representations requires a certain level of knowledge of music created using our system to manually created loops. We com- and proficiency in audio editing, which novices typically lack. pared both difficulty in creating loops and quality of the cre- ated loops. Our study suggested that using our system was significantly easier and faster than manual creation of loops and the quality of these loops were comparable to those created by experts. RELATED WORK Automation of tedious tasks that are inevitably part of the creative process, can allow users to focus their efforts on being creative [9]. Moreover, it can allow novices to be creative in ways that could be prohibitive based on their skills. This has previously been demonstrated on applications such as painting and sketching [2, 25]. Interactive music and audio editing systems have previously been built for various applications such as DJing [13], text- based speech editing [19], and interactive audio source separa- Figure 2. After clicking the Find me loops button, music loops extracted tion [5]. by LoopMaker are shown as orange rectangles with the rows ranked in order of confidence. The degree of confidence of each loop is indicated However, to the best of our knowledge, there is little existing by the opacity value of the corresponding rectangle. work in creating systems to assist in music loop creation from pre-existing music. Ong and Streich present an algorithm [17] to automatically create music loops. A fundamental limitation LOOPMAKER INTERFACE of their algorithm is that it creates loops only from segments We present the interface to our system in this section and the that repeat in a song. Although such segments can lead to underlying algorithms in the following section. Our system is 3 compelling loops, only a subset of all natural sounding loops a web application based on the Wavesurfer web audio API in a song follow this principal. They also present a system to with our algorithm running as a back end in Python through the 4 visualize existing loops [21] based on various criteria. Kitahara flask framework . First, the user uploads a song from which et. al [15] proposed a loop sequencer that automatically selects he or she would like to create loops and is then presented with music loops from a given set of existing loops based on a the interface that is shown in Figure 1. The waveform of the temporal evolution of excitement, which is specified by the given song is displayed in pink. user. Neither of these systems were created to assist in the loop creation process. Semi-automatic Mode In this mode, the user selects the approximate region of the The development of audio software has made it easier for am- desired loop, which is shown in green in Figure 1. The user ateurs to produce music [23]. Certain audio editing software then simply clicks on the Refine button and the system refines packages such as Ableton Live1 and Recycle2 provide seman- the selection boundaries to create a seamless loop. The user tic information and high level representations of music such as can listen to the loop in repeat mode by pressing the Play tempo and beat locations. Although this can make the process button. If satisfied with the result, the loop can be saved easier for users that are familiar with such representations, it by clicking the Export Region button. The user can create can still be a time-consuming process. Moreover, using these multiple loops in this way. 1https://www.ableton.com/en/ 3https://wavesurfer-js.org 2https://www.propellerheads.se/recycle 4http://flask.pocoo.org/ Automatic Mode In this mode, the user clicks on Find me Loops, and the system finds and displays the location of a number of loops that occur after the cursor position, as shown in Figure 2. The user can listen to a loop by clicking on it. Figure 4. Illustration of music segments. Letters a, b, and c represent different short segments in the song. Segment a and segment c are in Our system sometimes finds multiple variations of the same the same length (defined in beats).