AUTOMATIC EDITION OF SONGS FOR GUITAR HERO/FRETS ON FIRE Ana M. Barbancho, Isabel Barbancho, Lorenzo J. Tardon´ Cristina Urdiales Dep. Ingenier´ıa de Comunicaciones Dep. Tecnolog´ıa Electronica´ Universidad de Malaga´ Universidad de Malaga´ 29071 Malaga,´ Spain 29071 Malaga,´ Spain ABSTRACT the lower music tones. Also, it has to be considered that the button displayed on the left hand side of the screen (green In this contribution, an automatic system of song edition for button) is the most used in both games, however, the button at Guitar Hero/Frets on Fire is presented. The system performs the far right (magenta button) is the less used, actually, it only three fundamental stages: time analysis, frequency analysis appears in the higher difficulty levels of the game. and button allocation. The temporal analysis of the musical signal is used to obtain the rhythmic pattern of the song. The frequency analysis is perform on the basis of a Mel filter bank because these filters provide an approximation to the human perception of low and high frequency sounds. The allocation of buttons is done using the rhythmic pattern extracted, the Mel filters outputs and, also, taking into account a selectable difficulty level. The distributions of the buttons obtained with the designed system are similar to the ones found in the pre- a) b) determined songs of the games. Index Terms— Guitar game, rhythm pattern, Mel-filters. Fig. 1. a) Guitar Hero, b) Frets on Fire. In both, Guitar Hero and Frets on Fire, the player selects 1. INTRODUCTION a song from a predetermined list, without any possibility of including his favorite songs even having the audio file of the Guitar Hero is a popular rhythm game, developed by Red Oc- particular song and the rights to use them. To solve this prob- tane, in which a player uses a guitar-shaped controller with lem, Frets on Fire includes a song editor. This is not enough colored buttons to simulate rock music playing. More than 14 because the usage of the song editor turns out to be very bor- million copies have been sold of this game to date [1]. Frets ing, difficult and it needs a certain level of music knowledge on Fire [2] is an open source clone of Guitar Hero for PC that to obtain an acceptable result. The aim of this paper is to de- uses the keyboard as controller. This kind of games is so in- velop an automatic song editor for Frets on Fire/Guitar Hero teresting that specific controllers has been developed for blind so that the designed system generate from the audio file of any players [3]. song, all the necessary files to play the song in these games Principal screens of Guitar Hero and Frets on Fire are rep- with different difficulty levels. resented in Fig. 1. The goal in these games is the following: The outline of the paper is as follows: in section 2, the the player must press the correct combination of colored but- proposed system is presented, then the three main parts are tons synchronously with the music. The sequence of buttons described in the subsequent subsections devoted to time anal- is strongly related to the perception of tone and to the rhythm ysis (section 2.1), frequency analysis (section 2.2) and button of the music. These guitar games include several difficulty allocation (section 2.3). In section 3, some results are shown 3 4 levels that allow to adjust the number of buttons used ( , and, finally, some conclusions are drawn in section 4. or 5), the number of buttons played simultaneously and the speed at which the buttons have to be pressed. 2. PROPOSED SYSTEM The particular time in which a certain button has to be pressed is determined by the rhythm of the song. The button Frets on Fire requires five files to be available to play each to be pressed will depend, mainly, on the tessitura: the buttons song. These files are: on the right hand side of the screen will be assigned to the • label.png:Thisfile includes the song cover. higher music tones and the buttons on the left hand side to • song.ini:Thistextfile contains the song title, the artist This work was supported by the Ministerio de Educacion´ y Ciencia of name, the cassette color and the scores. the Spanish Government under Project No. TSI2007-61181. • song.ogg: This is the audio file of the song. 978-1-4244-4291-1/09/$25.00 ©2009 IEEE 1186 ICME 2009 Authorized licensed use limited to: Universidad de Malaga. Downloaded on September 25, 2009 at 08:37 from IEEE Xplore. Restrictions apply. • guitar.ogg:Thisfile includes guitar sounds. • Energy peaks separated by less than 0.1s, are removed. • notes.mid:Thisfile contains the information of the se- This is equivalent to restrict the highest beat rate to 600 quence of buttons to be pressed and for how long the beats per minute (bpm). buttons have to be pressed. • The duration of each beat is the smaller time between: the duration of time between two consecutive energy The core of the game is the sequence of buttons to press peaks and the elapsed time between the detection of a such that it is perceived as synchronized with the music. The peak and the window in which the measured energy is system proposed in this paper calculates, automatically and under a certain threshold relative to the peak energy. without any user interaction, the sequence of buttons to play Using these conditions, guitar rests are also taken into from the audio file of any song. account. INPUT Fig. 3 shows an example of the waveform, measured en- song.wav ergy and peaks selected from the initial passage of the Beat- les’ song ’Love Me Do’, together with its score. In this fig- ure, it can be observed how the selected beats match with the instants in which the signal amplitude increases significantly Time Frequency Analysis Analysis and with the line of percussion shown in the score of the song. 2.2. Frequency analysis The selection of the buttons to press is determined by the tes- Button Difficulty Allocation Level situra, as mentioned in previous sections. Therefore, in order to decide the buttons to push in the selected time instants, it is necessary to perform a frequency analysis of the song. OUTPUT In order to take into consideration the nonlinear perception notes.mid of frequency by the human auditory system, the Mel scale is Fig. 2. Block Diagram. used [5]. The Mel scale is mostly linear for low frequencies (< 500Hz) and logarithmic for high frequencies. Uniformly The block diagram of the proposed system is shown in spaced frequency bands are defined in the Mel frequency axis Fig. 2. According to this diagram, we consider three dif- to obtain the subjective information about the perception of ferent stages to solve the problem of generating a sequence low and high frequency sound. of buttons synchronized with the audio file: time analysis, to Now, an appropriate frequency resolution for the calcula- perform the rhythm analysis of the song (section 2.1), fre- tion of the power spectrum, before the application of the Mel quency analysis, to obtain the tessitura (section 2.2) and but- filters, must be selected. In our application, the signal is di- ton allocation, to create the button sequence from the rhythm, vided into windows of 46ms without overlap, and the power tessitura and difficulty level (section 2.3). In the following spectrum is calculated in each of them. subsections, these three stages are considered. In the next step, a bank of Mel filters is applied to the power spectrum [6]. We have considered a bank of 10 Mel 2.1. Time analysis filters ranging from 100Hz to 1KHz. The outputs of the five The objective of the temporal analysis of the audio signal is to even filters have been chosen, thus there is no frequency over- obtain its rhythmic pattern. The rhythmic pattern will deter- lap between the filters’ outputs. Each of these outputs will be mine the particular time instants at which the different buttons related to a specific button in the game. The frequency range have to be pressed and how long they have to be pressed. The analyzed has been chosen taking into account that, in pop and rhythm of a musical passage of a song is straightly related to rock music, most of the musical and rhythmic information the points where a significant energy peak appears [4]. are contained in the low frequencies and these are the genres Therefore, in the proposed system, rhythmic patterns are usually played in these applications. Also, note that, this fre- obtained finding significant energy peaks in the audio signal quency range is not directly related to the guitar frequencies. in the time domain. Each peak detected corresponds to a mu- Fig. 4 a) shows the raw values obtained directly at the sical beat. The detection process is as follows: selected outputs of the bank of Mel filters applied to the initial • Music signal energy is calculated in windows of 5ms of fragment of the song ’Love me do’ by The Beatles. Note duration, without overlap. In our evaluations, we have that the amplitude of the filters decreases while the number of found that this is an appropriate temporal resolution. the filter increases because the energy is concentrated on the lower frequencies. • Peaks over a certain threshold are detected. These peaks are considered to correspond to music beats. The 2.3. Button allocation threshold is set to the half of the maximum energy The selected outputs of the Mel filters will be employed to peak.