FOMUS, a Music Notation Software Package for Computer Music Composers

David Psenicka School of Music, University of Illinois [email protected]

Abstract MUS (FOrmat MUSic) offers an alternative to having to deal with these limitations, providing algorithms that make de- FOMUS is a software tool that facilitates the process of rep- cisions regarding note spellings, voice separation, rhythmic resenting algorithmically generated data as musical notation. spelling, etc.. It organizes the results into a format suitable It does this by spelling notes, quantizing offsets and dura- for processing by a graphical notation program such as Lily- tions, laying out information into voices and staves and mak- Pond (Nienhuys and Nieuwenhuizen 2003), Common Music ing many other decisions such as where to include clef changes Notation (Schottstaedt) , Finale or Sibelius. Importing into or octave transposition markings. A computer music com- Finale and Sibelius is done via the MusicXML file format poser using this software can work while being less concerned (Good) . Once saved or imported into one of these programs, with the tedious difficulties involved in importing data into a much less work is required to adjust the results into a finished, graphical notation program. Information such as dynamic notated score. markings and articulations can also be generated and made FOMUS is available for free at to appear in the score. The program is written in Lisp, inter- http://common-lisp.net/project/fomus/ and is licensed under faces with Common Music and outputs files suitable for ren- the LGPL. It currently runs in SBCL, CMUCL, LispWorks dering with LilyPond or CMN or importing into a program and Allegro CL in the Linux, OS X and Windows operating that reads MusicXML format files such as Finale or Sibelius. systems. It interfaces with Common Music (Taube 1997) and is also included with the PWGL/ENP visual programming en- 1 Introduction vironment (Laurson and Kuuskankare 2002). Converting data generated by computer algorithms into 2 Related Work musical notation can be a tedious procedure that is prone to error. Conventions exist in musical notation that are often not Solutions to several of the problems mentioned in this pa- a concern for composers until it is time to render informa- per such as automatic note spelling and voice separation have tion into a notated format suitable for musicians to read and been proposed. A dissertation by Jurgen¨ Kilian (2004) sum- interpret. Such conventions might include barlines, note ac- marizes several approaches to quantization, note spelling and cidentals, rhythmic spelling, and other elements that are typ- voice separation (or “streaming”) and describes his own sys- ically too difficult or complicated to bother with when the tem for solving these issues. The note spelling algorithm de- main concern is generating information more directly related scribed here is similar to an algorithm developed by Emil- to audible results. ios Cambouropoulis (2001) which bases note spelling deci- A typical method of importing data into a popular graphi- sions on minimal use of accidentals and avoidance of un- cal notation program such as Finale or Sibelius might involve common intervals such as augmented and diminished ones. using MIDI files to import information into that program. A paper by David Meredith (Meredith 2003) offers a sum- This procedure is rather limited, since MIDI information is mary of this and several note spelling techniques including constrained to pitch and duration and contains no provisions his own. Approaches to voice separation are found in Cam- for other important notational elements such as articulation bouropoulis (2000), Chew and Wu (2004), Kilian and Hoos and dynamic markings. The MIDI import features for these (2002), and in the The Melisma Music Analyzer program by programs are adequate at best, and might not properly import Daniel Sleator and Davy Temperley (Sleator and Temperley) . rhythms more complex than a triplet. Note spellings are usu- FOMUS differs somewhat from these algorithms in that it is ally much simpler than what is desirable. Much work may meant to provide a general solution for any input data that also be involved in double-checking them to insure that the is encountered, whether it represents music that is sparse or score is accurate. dense, polyphonic or homophonic, tonal or atonal, etc.. It is Most other methods of importation involve similar limita- a practical application designed to make the composer’s task tions or require the user to specify detailed elements such as easier rather than provide accurate transcriptions of classical accidentals, tied notes, or barlines that are too cumbersome or popular music. to deal with until one is working directly with the score. FO-

75 3 Features attached to any note event. The EXEC function then sends the information to FOMUS for processing and outputs a file for FOMUS automates the following tasks: importing or compiling using one of the notation programs mentioned above (in this case, LilyPond). Users can also send • Quantizing (choosing rhythms and tuplets that mini- information to FOMUS via Common Music’s EVENTS func- mize the amount of error between input offsets and no- tion. The same information can also be stored in a text file tational offsets) which is then processed by FOMUS, an option for users who • Note spelling (semitones and quarter-tones are currently don’t want to program in Lisp or use Lisp syntax to specify supported) events.

(fomus-init :output ’(:lilypond :view t)) • Distribution of notes into voices and staves (fomus-newpart 1 :name "Piano" :instr :piano) (fomus-newtimesig :off 0 :time ’(5 8)) (fomus-newnote 1 :off 0 :dur 3/2 :note 66 :marks ’(:marcato)) • Proper note splitting and tying (rhythmic spelling), tak- (fomus-newnote 1 :off 3/2 :dur 1/2 :note 67 :marks ’(:staccato)) (fomus-exec) ing into account meter subdivisions, dotted rhythms, and special cases such as syncopated rhythms Figure 1: FOMUS Input Example • Part grouping and ordering (given the instruments for each part and an ensemble type such as an orchestra) FOMUS decides whether to divide the 5/8 measure into • Transpositions and checking of instrument ranges 3+2 or 2+3 eighth notes, whether to spell note 66 (F#/Gb above middle C) with a sharp or flat, and which staff or staves • Decisions regarding placement of clef changes and oc- to place the two notes on. Figure 2 shows the result of exe- tave transpositions cuting the contents of Figure 1. • Layout of articulations, slurs, beams, dynamics and text markings, and other notational elements • Many other minor but helpful tasks (beaming, applying cautionary accidentals, etc.)

• Output to several different file formats: LilyPond, CMN, Figure 2: LilyPond Output MusicXML and MIDI. The program is designed with modularity in mind so that any of these operations will eventually be performed by choos- 3.2 Quantizing ing an algorithm from among several similar ones. Users will FOMUS quantizes note offsets and durations by searching be able to choose the note spelling algorithm that gives the through all possible combinations of rhythms and tuplets in most desirable output for their own work. Many of these op- each measure and finding the ones that deviate the least from erations can be turned on or off if the user wishes to make the user’s input values. This is done according to a quan- certain decisions him/herself. Many automated decisions can tize duration and maximum tuplet value specified by the user. also be overridden, in which case FOMUS uses and incorpo- Note durations that are quantized to a value of zero become rates the user’s information when making its own decisions. grace notes. The user may also specify whether to minimize The following subsections give an overview of how FO- the average error or the least-mean-squared error between in- MUS handles some of these tasks. put and notation. One may also specify exact offsets and du- rations in the form of integers and ratios, in which case no 3.1 Input quantizing is necessary and rhythms are spelled exactly as Data is input either through Lisp expressions or a text file. specified. Figure 1 shows a simple example using Lisp expressions. The The search process described above is similar to the search INIT function in the first line specifies that the output file is used in the following subsection on rhythmic spelling and to be a LilyPond file and that it is to be viewed immediately. uses the same rules and constraints for determining what is Many other global options may be specified in this line. The or isn’t allowed. Each measure is recursively divided into NEWPART function specifies a part with an ID of 1 for notes smaller and smaller segments and the best solutions (the ones to be attached to. The :piano symbol tells FOMUS that it is with minimal quantization error) are combined into larger so- to notate a part for a piano instrument using a grand staff with lutions to form a complete solution. treble and bass clefs and octave transposition signs where appropriate. Two notes are then specified–the first a dotted 3.3 Rhythmic Spelling eighth note and the second an eighth note (the default beat One of FOMUS’s basic operations is to decide on how duration is a quarter note, which can be changed). The mar- to split and tie notes within a measure so that the result is cato and staccato indications are examples of “marks,” sym- properly notated according to the given time signature. Rests bols that designate a variety of articulations, ornaments, dy- are added automatically. The user must specify time signature namic markings, and special notation instructions that can be 76 change events which contain information such as what note piano) these are permitted and converted into chords as appro- duration equals a beat, how measures are to be divided (a 7/8 priate. Scoring is based on whether a note event is higher or measure, for example, may be divided into either 3+2+2 or lower than surrounding note events in other voices, whether 2+2+3 or may always be divided into 2+3+2), and whether or a note is simultaneous with, overlapping, or adjacent to notes not the time signature represents a compound meter. FOMUS in the same or other voices, how smooth the “voice leading” uses this information to determine the simplest combinations is, how closely spaced together the chords are, and how “well of ties and tuplets that precisely represent the rhythms from balanced” the voices are (the algorithm tries to put an equal the quantizing stage mentioned above. Decisions regarding amount of material in each voice). metric subdivisions are made consistently across all parts, and FOMUS also lays out voices onto multiple staves, if re- special cases are also accounted for (for example, syncopated quired, adding clef signatures and octave transposition signs patterns in the form of quarter-half-quarter or eighth-quarter- when necessary. This is a separate algorithm but is again done quarter-eighth). Nested levels of tuplets are also possible. by searching through possible solutions and using scores to The rhythmic spelling algorithm works by dividing each find the best result. Scoring in this case is based on the num- measure recursively into smaller and smaller segments and ber of clef changes that occur in close proximity, the number finding the simplest valid combination of notes, ties and tu- of staff changes in close proximity, the number of simultane- plets in each segment. As smaller solutions are found they ous voices on each staff, and the current “clef order” (a bass are combined into larger solutions to form a complete solu- clef in an upper staff simultaneous with a treble clef in a lower tion for the entire measure. staff receives a severe penalty, for example). Once rhythmic spelling decisions have been made, FO- MUS generates beaming information so that the output prop- 3.6 Other Features erly represents the decisions made regarding metric subdivi- sions. FOMUS contains many other features, such as a database of common orchestral instruments and the information neces- 3.4 Note Spelling sary for notating them properly (for example, the clefs used, their transpositions, pitch ranges, etc.). Users may specify Note spellings are determined by searching through all one of these default instruments or define their own. Per- the note events in each part and choosing the accidentals that cussion instruments with little sustain and other special artic- score the highest in terms of forming easy-to-read diatonic ulations (such as pizzicato strings) are notated with simpler intervals and appropriately spelled ascending or descending durations that are easier to read. Time signature changes are lines (ascending lines with sharps or descending lines with automatically adjusted to avoid awkward “1/8” or other such flats). Quarter-tone spellings are also possible, in which case meter changes that might occur. Multiple voices within a part scoring also additionally depends on whether similar inflec- are combined into chords where appropriate. Also, some ar- tions (quarter-tone sharps or flats) appear in close proximity. ticulations and ornament marks are treated specially, such as Users may also provide their own note spellings, overriding mordents or trills with accidentals above or below them (FO- FOMUS’s decisions. Scales and key signatures are not yet MUS notates them with the correct accidental when the user included in the algorithm’s decision making, though this will supplies the second pitch), pizzicato and arco markings (FO- be added in future versions of the program. MUS sorts these out automatically) and artificial harmonics. The search process described above allows decisions to be made over extended sequences of notes or chords. FOMUS 3.7 Backends backtracks and evaluates different possibilities until it finds the best combinations. A spelling decision takes into account Four backends are currently supported: LilyPond, CMN, all notes within a certain distance (calculated exponentially MusicXML and MIDI. The LilyPond (Nienhuys and Nieuwen- with respect to both duration and pitch) and weighs all scores huizen 2003) typesetting program is excellent for immedi- according to their distances. A similar kind of search is used ately processing and displaying notation that is well format- in the voice and staff separation algorithms described below, ted and readable. Common Music Notation runs natively in and can be adjusted to produce better quality output at the Lisp, and is capable of displaying all of the same notational expense of computation speed. elements. MusicXML (Good) is suitable for importing nota- FOMUS also provides algorithms for handling caution- tion into commercial notation programs like Finale, Sibelius ary accidentals and other details such as how accidentals are and Igor Engraver. A MIDI backend is available via Com- handled when they appear beneath octave transposition signs. mon Music and is useful because it is fairly customizable. The user can provide his/her own functions to translate no- 3.5 Voice and Staff Separation tation into MIDI output, thus “personalizing” the output to a great extent to produce a better rendering than what other A similar type of search is used for separating voices within programs might produce on their own. a part. Choices are made in the context of note events and Figures 3 and 4 show how FOMUS notates a short, ran- chords occurring within a certain temporal distance. Overlap- domly generated piece. The Lisp expression contains a loop ping voices are always separated if this is not allowed for the structure that collects random note events and passes them to given instrument, though for some instruments (such as the

77 the FOMUS function for processing. The output was gener- forced to work within the bounds of that method. Using MIDI ated on the fly with LilyPond. files as a means of transporting data, for example, encourages composers to only work with pitch and duration, and may in- (fomus :backend ’(:lilypond :view t) fluence the types of rhythms that are composed depending on :ensemble-type :orchestra :default-beat 1/4 how easily they can be imported into a certain notation pro- :global (list (make-timesig :off 0 :time ’(3 4)) (make-timesig :off 7 :time ’(5 8))) gram. FOMUS is designed to remove many of these limita- :parts (list (make-part tions and make important aspects of notation easily accessible :name "Piano" :instr :piano and controllable within a computing environment. The com- :events (loop poser can then focus less on how to arrange his/her score and for basenote in ’(54 42) nconc (loop for off = 0 then (+ off dur) more on the process of making music. and dur = (/ (1+ (random 4)) 2) while (< (+ off dur) 12) collect (make-note :voice ’(1 2) :off off :dur dur References :note (+ basenote (random 25)) :marks (when (= (random 2) 0) Cambouropoulos, E. (2000). From MIDI to Traditional Music ’(:accent)))))))) Notation. In Proceedings of the AAAI Workshop on Artificial Intelligence and Music: Towards Formal Models for Compo- sition, Austin, Texas, pp. 174–177. Figure 3: FOMUS Input Example Cambouropoulos, E. (2001). Automatic Pitch Spelling: From Numbers to Sharps and Flats. In Proceedings of the VII Brasilian Symposium on Computer Music, Fortaleza, Brasil, pp. 174–177. Chew, E. and X. Wu (2004). Separating Voices in Polyphonic Music: A Contig Mapping Approach. In Computer Music Modeling and Retrieval: Second International Symposium, pp. 1–20. Figure 4: LilyPond Output Good, M. MusicXML in Practice: Issues in Transla- tion and Analysis. Last Viewed March 10, 2006, at http://www.recordare.com/good/max2002.html. Kilian, J. (2004). Inferring Score Level Musical Information 4 Conclusion and Future Work From Low-Level Musical Data. Ph. D. thesis, Technischen Universitat¨ Darmstadt. FOMUS has potential for uses other than music compo- Kilian, J. and H. H. Hoos (2002). Voice Separation - A Local Op- sition. Coupled with Common Music, it is an excellent tool timisation Approach. In Proceedings of the 3rd International for immediate viewing of musical data stored in the MIDI file Conference on Music Information Retrieval, pp. 39–46. format. Its modular structure allows for the addition of func- Laurson, M. and M. Kuuskankare (2002). PWGL: A Novel Vi- tionality that would be useful for music transcription. For sual Language Based on Common Lisp, CLOS and OpenGL. this reason, modules for beat, meter and key detection are In Proceedings of the International Computer Music Confer- planned for future releases, as well as more specialized algo- ence, Gothenburg, Sweden, pp. 142–145. rithms for note spelling and voice separation. Following are Meredith, D. (2003). Pitch Spelling Algorithms. In Proceedings some planned additions and improvements: of the Fifth Triennial ESCOM Conference, Hanover Univer- sity of Music and Drama, Germany, pp. 204–207. The Soci- • Combining of separately notated sections into one score ety for Music Perception and Cognition. • Support for polymeters Nienhuys, H.-W. and J. Nieuwenhuizen (2003). LilyPond, a sys- tem for automated music engraving. In Proceedings of the • Support for proportional notation XIV Colloquium on Musical Informatics, Firenze, Italy. Read, G. (1979). Music Notation: A Manual of Modern Practice • Better support for microtonal pitches (depending on (2nd ed.). New York, NY: Taplinger Publishing Company. how these are supported in the backends) Schottstaedt, B. Common Music Nota- tion. Last Viewed March 10, 2006, at • Algorithms for detecting meter divisions and tonal ar- http://ccrma.stanford.edu/software/cmn/cmn/cmn.html. eas for automatic placement of time and key signatures Sleator, D. and D. Temperley. The Melisma Mu- • Support for more backends including SCORE, Guido sic Analyzer. Last Viewed March 10, 2006, at and ABC. http://www.link.cs.cmu.edu/music-analysis/. Taube, H. (1997). An Introduction to Common Music. Computer The intention of FOMUS is to remove the difficulty in- Music Journal 21(1), 29–34. volved in rendering algorithmically-generated data into read- able music notation. When relying on methods of import- ing notational data that only support limited features (such as the case of using MIDI files described above), a composer is

78