1 A model based approach to automatic harmonization of a melody H. V. Koops [email protected] Abstract|HarmTrace has shown to be an effective model is not a guaranteed way of making good music, it merely to derive harmonic functions of chords in their tonal con- tells us how music was written in a specific timeframe of text. This article describes the addition of a chord genera- tion and selection mechanism that investigates the abilities our musical history. of the HarmTrace model as an automatic harmonizer that The practice of abstracting rules of harmony and placing generates chords with a given melody. A system which is them in a linguistic framework has been a part of computer able to generate multiple harmonically well-formed chord sequences for a given melody is added to the model. From science at least since the sixties of the previous century. the generated sequences the best one is chosen, by means Work by people like Jackson[1] and Winograd[2] already of smallest amount of parser errors. One experiment is showed that there is a strong connection between the ba- carried out in which chord sequences for carefully selected melodies are generated. Subsequently, in a survey a panel sic ideas of linguistics and the structure of music, by cre- of harmony experts are asked, based on listening to the ating a grammar that describes the functional elements of model generated sequences, to describe their professional a harmonic progression. opinion and rating of these chord sequences. A second ex- periment is carried out in which a chord sequence for an Automatic harmonization is a natural extension (and selected melody is generated, and compared to the original application of) harmonic analysis, and an important com- accompaniment of that melody that is considered a ground ponent in music information retrieval (MIR). Its function truth. Unlike all currently existing systems, this system is to clarify principles used by composers and musicians, guarantees the generation of well-formed chord sequences, in which the constituents are automatically functionally ex- and to capture these in an artificial intelligence (AI) frame- plained with regard to their context. From the experiments, work. Researching the mental steps involved in the process it is shown that the system is capable of generating correct of harmonization and automating these is valuable contri- chords per melody tone. However, it is also shown that harmonizing a melody with individually well-formed chord bution to the study of music and AI, and MIR. With the sequences from the HarmTrace grammar do not guarantee a advent of ever increasing computational power, a multitude harmonically well-sounding coherence between the sequence of computerized harmonization systems have been brought and the melody. This can be explained by the nature of the forward to automate the process of generating additional system, which built to analyze, rather than to create music. voices to a given melody. In addition to rule based gram- Index Terms|Harmony, Automatic harmonization, mar systems, statistical systems based on hidden markov Haskell, Haskore, HarmTrace models[3] and genetic algorithms[4] have been brought for- ward. The rule based system HarmTrace is made in the func- I. Introduction tional programming language Haskell. Given a sequence of VER since the Middle Ages, when singers in monaster- symbolic chord labels, HarmTrace automatically derives Eies began to experiment with the addition of another the harmonic function of a chord in its tonal context[5]. voice to the chant, tentative efforts are made of discover- Among other applications, these functional annotations ing laws that accurately describe the way multiple voices can be used to improve the quantification of harmonic sim- should sound together. The aspiring human mind that ilarity of two annotated chord sequences.[6] wants to know and understand the laws of art brought Haskell[7] is a purely functional programming language forward theories to capture this phenomenon of harmony. with a very active research community. It is a statically With time, these laws have been changing subject to fash- typed, lazy functional programming language which allows ion, politics and even ecclesiastical law. With every gener- for a very elegant and concise programming style. Instead ation, new laws are created of the given phenomena of art, of a computation in terms of statements that change a pro- some of which are rediscovered, rewritten or changed. gram state, a functional language treats a computation as The hunt for a final theory of harmony is an honest the evaluation of mathematical functions and avoids state search that involves mental gymnastics of finding ways of and mutable data. Functional programming has its roots rediscovering ourselves as a species that creates art. How- in lambda calculus, a formal system developed in the 1930s ever, it is a false conclusion that if laws can be derived from to investigate function definition, function application, and the current musical phenomena that it will hold for future recursion. Many functional programming languages can be phenomena as well. Even worse is concluding that this viewed as elaborations on the lambda calculus.[8] could be used as a measure for aesthetics. For harmony Because of the availability of good error-correcting theory can be seen as the collected and systemized deduc- parsers, and algebraic datatypes to implement context free tions gathered by observing the practice of composers over grammar-like structures, Haskell is a good and efficient a long time, describing what has been common practice. It choice to implement rule-based functional harmonic anal- 2 and upper case letters (A { G and a { g ), and the sub- and super-prime symbols (;) and (0) to describe each individual tone. When a tone is described as a sign in a notation sys- tem, it is called a note. Notes are a combination of pitch and duration, rather than just pitch. This basic element governs melody and harmony. In musical Set theory, pitch classes are defined by two Fig. 1. A typical chord sequence and its harmonic analysis (as equivalence relations. Pitches belong to the same class if generated by HarmTrace) The chord labels are printed below the they have some relation of compositional or analytical in- score, and the scale degrees and functional analysis above the score. terest, for example, the octave relation[11]. The second re- Voice-leading is ignored for simplicity. In this case, the input lation is the enharmonic equivalent relation, which means was a string consisting of the key of C and a string of chords "C:Maj,C:Maj,F:Maj,D:Sev,G:Sev,C:Maj". Figure taken from Ma- that all the pitches played on the same key of an equal- galh~aesand Haas (2011) [9] tempered keyboard are in the same set. From these two equivalence relations there are just 12 pitch classes, corre- sponding to the notes of the chromatic scale, often num- ysis. The collection of Haskell datatypes in HarmTrace bered from 0 to 11. The choice of which pitch class to can be viewed as a very powerful context free grammar call 0 is a matter of convention or expedience. This paper (CFG). The language defined by this CFG consists only sticks to the commonest conventional choice of calling C 0, of the combined values that match the structure of the in which case C] is 1, D is 2 and so on. datatype. HarmTrace's grammar defines the language The distance between two notes, measured by their dif- of well-formed chord sequences, because the chords are the ference in pitch, is called an interval. An interval can be values, and the datatype represents the relations between melodic or harmonic, in which the notes that make up that the structural elements. interval sound consecutively or together, respectively. In This article shows how a system that can complement a Western tonality, an interval is commonly classified as a melody with a chord sequence can be added to Harm- combination between its quality (major, minor, perfect) Trace. After generating multiple chords for melody and number (unison, second, third, etc.). The number tones, a statistical system creates a multitude of progres- represents the difference in the number of staff positions sions. These progressions are subsequently parsed with the between two notes, and the quality derives its name the HarmTrace, to derive their functional structures. From fact that some intervals come in different sizes. An exam- these progressions, the best one is chosen by means of the ple is the third. It is called a third because it takes up three least amount of errors. This harmony is combined with the staff positions, but can be major or minor, depending on input melody to create a MIDI file as an audible output. whether it consists of three semitones, or four. The smaller Before writing the MIDI file, an algorithm that determines interval is called a minor third, and the larger one a major the least varying inversions is used to diminish jumps in third. A perfect fifth is a difference of seven semitones, and the harmony. an octave spans twelve semitones. An example of the most Simplified Haskell code examples are shown where they common intervals and their qualities are listed in figure 2. clarify the text, but no extended knowledge of the Haskell min per syntax is required to understand these. The first section of min maj min maj per per min maj maj this article introduces basic music theory. The second sec- tion explains a basic harmony theory and a classic way to unison 2nd 2nd 3rd 3rd 4th 5th 6th 6th 7th 7th octave manually harmonize a melody. Section four explains the technical workings of the automatic HarmTrace Har- Fig.