Earnest 1

MIDI Controller Designs:

The Best Options for Performing Live Digital Music

By

Joey Earnest

Research Procedures in Music

Dr. Brown

Fall, 2013

Earnest 2

The sound of music has been shaped by mechanics, culture, and biology. Modern technology has changed the way in which music is physically performed and composed. In the world of digital music, tools like midi-controllers give performers control over a wide array of musical parameters, in real time. Highly refined computer programs can make this process even more streamlined. Traditionally, keyboard styled midi and controllers have ​ ​ been the standard note input devices for production and performance purposes. Considering that the actual design of the piano keyboard must accommodate physical sound-making mechanisms, such as strings and chimes, one can see its legitimacy as an acoustic “controller.” However, a digital platform, such as an isomorphic keyboard, eliminates the need for the controller to be laid ​ ​ out over several feet.

The traditional piano keyboard, although ergonomic in many respects, does display limitations. For example, the piano-keyboard only utilizes left to right and to a limited degree, up and down, while humans have the ability to move their hands in many directions along three axes, including up and down, left to right, forward and back. The only reason piano keyboards are still relevant as midi controllers is because of the traditionalism within various musical communities that require a piano style keyboard to reproduce a specific idiom, or have been trained in that method. Isomorphic midi-controllers and other accelerometer or sensor based controllers that utilize all three spatial dimensions, are superior in design to traditional piano style controllers for the transmission of digital music data. Theoretically, all of the same music of the piano can also be played on an isomorphic keyboard. This begs the question: if a musician wants to make exclusively electronic music, what benefit would learning the piano have? Gustav

Mahler would say, “Tradition is not the worship of ashes, but the preservation of fire” (Mahler, Earnest 3

1860-1911). Frank Zappa, on the other hand said, “Progress is not possible without deviation”

(Zappa VPRO interview Netherlands, 1971). These perspectives are at the heart of this inquiry, and pose the question, what are the advantages and disadvantages of using newer isomorphic styled controllers as opposed to traditional piano keyboard controllers? This inquiry will find both advantages and disadvantages of the piano keyboard and isomorphic keyboards in contemporary music by focusing on their mechanical, cultural, and biological contexts.

The musicologist, Bart Hopkin, provides insightful information on building all kinds of instruments. Understanding the history and mechanics of instrument design is a foundational aspect to the argument of the piano keyboard being a subpar device to transmit midi-data. In his book, Musical Instrument Design: Practical Information for Music Making, Bart Hopkin ​ ​ provides a brief history of the development of musical keyboards in Europe:

The layout of the standard keyboard - the familiar pattern on five raised black keys and seven white keys to the octave - was never invented or planned in any clear sense. It came into being in an evolutionary process spanning several centuries. The earliest keyboard-like mechanisms were sets of broad, flat sliders protruding from the fronts of organs of Greek antiquity. By the Middle Ages, the norm was a set of seven sliders or levers comprising the diatonic octave, lying in a single even row. Beginning around the thirteenth century, progressively more levers appeared to accommodate chromatic tones, with the the additional keys distinguished from the original diatonics by their placement in a second row above. By the fifteenth century, an arrangement much like the present one had become widespread, with the seven below and five above- but still, at that time, in the form of rows of levers. In the course of the fifteenth and sixteenth centuries this evolved into the bed of ivory and ebony that we see today (Hopkin & Scoville, 2010).

Traditionally, the developments of the keyboard mirrors the developments in music theory, such as the understanding of the harmonic series, changes, polyphony, equal temperament, and chromaticism. Contemporary isomorphic keyboard designs may eventually open up new creative possibilities for changing between a multitude of newer tonal systems, including microtonal Earnest 4

systems, mid performance. Hopkin mentions that many contemporary keyboard designs “depart more radically from the existing standard” (Hopkin & Scoville, 2010).

A variety of hexagonal shaped layouts, including Erv Wilson’s Bosanquet Keyboard ​ ​ ​ design, exemplify this shift. Hopkin says, “One important idea has been to use two-dimensional arrays. This brings more keys into a smaller area, allowing the player to reach more tones with less stretch. It also allows a more sophisticated approach to pitch relationships, since one can work with vertical and diagonal spatial parameters as well as horizontal” (Hopkin & Scoville,

2010). When these newer tessellating isomorphic keyboard designs are used in a digital context, the creative potential increases dramatically. When the keyboard is comprised of a grid of buttons or touch sensitive pads triggering digital signals, as opposed to a series of levers and hammers hitting large metal strings, it becomes easier to design small compact keyboards with more notes. It also means the integrity and efficacy of the keyboard layout does not need to be compromised by the inherent physical limitations of acoustic instrument design. The digitization of keyboards only adds to the relevance of this idea. Since the cumbersome, large mechanical systems necessary for acoustic instruments to make sound are no longer a significant factor in the design of MIDI keyboards, it raises the question, why aren’t isomorphic MIDI keyboards more popular than piano-style MIDI keyboards for playing polyphonic passages of music? The

Hexagonal design makes horizontal and diagonal moment more fluid; vertical leaps conveniently shift octaves. The “Hex Keyboard” is a good substitute for the piano keyboard because it can also easily reproduce idiomatic piano passages.

Isomorphically designed keyboards can be far superior at transmitting MIDI data than traditional piano keyboards for several reasons. Most importantly, the learning curve is faster on Earnest 5

an isomorphic keyboard because musical patterns and note groups can be replicated in multiple transpositions without altering the physical shape of the group. Another important reason is space and efficiency. Isomorphic keyboards are compact and can fit many notes into a tight grid rather than sprawling out several feet. There are also the ergonomic issues surrounding keyboard design. Isomorphic keyboards have less potential to cause physical performance injury than piano keyboards. The physical nuances of the piano keyboard have affected the literature.

Although many composers certainly push the boundaries of what is possible on any given instrument, the physical limitations of each instrument inevitably shape the literature. Many of the greatest piano pieces ever written, can be played with ease and comfort by competent piano players. The largest advantage to playing on a traditional keyboard is to reproduce traditional idiomatic literature. However, clinging to the piano keyboard (as a midi controller) for reasons of comfort and familiarity can potentially slow the process of innovation in a constantly evolving digital music world.

The Computer Music Journal presents research out of MIT suggesting that there are ​ ​ specific advantages to using isomorphic style keyboards in comparison to traditional keyboards.

One such article discusses various designs or layouts of the keyboards, as well as the way in which we identify intervals in music. “This article answers this question by presenting examples of two related tuning continua (parameterized families of tunings where each specific tuning corresponds to a particular value of the parameter) that exhibit tuning invariance (where, on an appropriate instrument, all intervals and chords within a specified set have the same geometric shape in all of the tunings of the continuum)” (Milne, Sethares, & Plamondon, 2007). The article states “Microtonality presents interesting challenges, not only for the design of new instruments Earnest 6

(and, of course, for performers of traditional instruments), but also for notation. Composers have adopted different, often idiosyncratic, conventions for naming the pitches of nonstandard tuning systems and displaying them on the page” (Milne, Sethares, & Plamondon, 2007). The article discusses the mathematical relationships of different frequencies and harmonics in various tuning systems and temperaments. It also talks about geometry and symmetry and how it relates to isomorphic keyboards. The research presented by the MIT study concludes that “isomorphic keyboards can be coupled with appropriate computer technology to provide the deeper benefits of tuning invariance and dynamic tuning” (Milne, Sethares, & Plamondon, 2007). In addition:

Music education, performance, and composition can benefit from the transpositional and tuning invariance provided by isomorphic button fields. The piano-style keyboard is incapable of achieving the advantages of an isomorphic keyboard for three reasons. First, it is essentially one-dimensional, and one-dimensional layouts can only have transpositional invariance for equal temperaments. Second, its offset pattern of twelve keys per octave is only well suited to tunings that use twelve (or fewer) notes per octave. Third, the piano keyboard’s offset white and black keys hide the consistent patterns of 12-TET—let alone any other tunings—forcing students to learn each key’s scales, chords, and other properties by rote.

In contrast, with an isomorphic button-field, a student need only learn the geometric shape of a given interval once (within a given temperament), and thereafter apply that knowledge to all occurrences of that interval, independent of its location within a key, across keys, or across tunings. This reduces rote memorization considerably and engages the student’s visual and tactile senses in discerning the consistency of music’s patterns (Milne, Sethares, & Plamondon, 2007).

The article notes significant advantages in isomorphic keyboards’ ability to transition seamlessly between a variety of tuning systems. The study suggests that, “A student, having mastered the use of an isomorphic button field in any one of these tunings, can apply the same motor skills and music theory to all the rest, making multicultural music education considerably easier”

(Milne, Sethares, & Plamondon, 2007). However, the authors did concede that “there are also Earnest 7

disadvantages to such isomorphic keyboards compared to that of the piano. The playing of diatonic intervals connected by parallel motion is more complex than on the white keys of the piano, and with some layouts, there is a less obvious linear relationship between pitch height and location on the keyboard. Although isomorphic keyboards have been proposed and built in the past, they do not have a large installed user base or a deep repertoire” (Milne, Sethares, &

Plamondon, 2007).

In order to fully explore the advantages and disadvantages of these keyboards, it is important to clearly define what it means to be an isomorphic keyboard. The Oxford Dictionary defines isomorphic as “1. Corresponding or similar in form and relations. 2. Having the same ​ ​ crystalline form” (“Isomorphic,” 2018). For the purposes of this research, this definition falls short for describing the specific geometric patterns most common in isomorphic keyboard designs. This essay ultimately examines tessellating isomorphic keyboards. According to the ​ ​ Oxford Dictionary, tessellation is “An arrangement of shapes closely fitted together, especially of polygons in a repeated pattern without gaps or overlapping” (“Tessellation,” 2018). In a conference paper titled ISOMORPHIC TESSELLATIONS FOR MUSICAL KEYBOARDS, the ​ ​ authors give specific examples of tessellating isomorphic keyboard layouts:

There are three regular tessellations that can be formed on a plane. They are built upon three regular polygons: the square, the hexagon, and the equilateral triangle. Of the three tessellations, only the square and hexagon tilings are unidirectional. Equilateral triangle tilings are inherently bidirectional, in that, if you have a triangle that is pointing up, you require a triangle that is also pointing down in order to form a complete tiling. The bi-directionality of triangular tiling violates true isomorphism, as identical chords will appear to have different shapes due to the directionality component of the triangles. It is this feature of triangular tiling that nullifies its use in isomorphic keyboards (Maupin, Gerhard, and Park, 2011).

Earnest 8

Traditionally, “Most stringed instruments use a square tessellation as the basis of tuning. The horizontal direction is always chromatic, where each fret represents a semitone. The vertical direction is often tuned to a perfect fourth or perfect fifth” (Maupin, Gerhard, and Park, 2011).

Of course, this pure isomorphism is not always expressed perfectly. For example, in standard guitar tuning, the 5th string B has been lowered by a semitone to assist the performer with their ability to play chords with the 6th string also known as barre chords. “In this case, a non-isomorphic tuning scheme is adopted to allow for more comfortable, moveable isomorphic chords” (Maupin, Gerhard, and Park, 2011). It is important to note that the tuning of the guitar has been altered to accommodate tonal harmony in addition to the natural ergonomic issues that arise from the asymmetry of the human hand.

A variety of popular tessellating square keyboard designs exist on the market for digital musicians. There are clear creative, theoretical, and performance advantages to using these devices, especially when it comes to creating live electronic music. These include MPC controllers like the MPC500 and Native Instruments’ Maschine, and other advanced MIDI ​ ​ ​ ​ controllers like the Ableton Push, Push 2, and the Novation LaunchPad. The square grid makes ​ ​ ​ ​ ​ ​ simultaneous drum sequencing and clip triggering a possibility. The relationship that Ableton ​ Live has with its Push controllers exemplifies these advantages. The Push controller can switch ​ ​ ​ ​ ​ back and forth between a drum machine with a beat pad, step sequencer, and track selector all simultaneously visible, and a fully functional isomorphic polyphonic keyboard. The performer can go from programming and performing beats live, to playing melodic and harmonic material on various permutations of tessellating square note layouts. Performers can also choose between several note layouts that emphasize various diatonic and non-diatonic scales, and the most Earnest 9

flexible chromatic layout. It should also be noted that interfacing with these square controllers can be surprisingly intuitive. Patterns and groups are more consistent and easily identifiable than on traditional piano keyboard.

Selections from The Biology of Musical Performance and Performance-Related Injury ​ explain the physical symptoms of chronic music playing and injuries resulting from improper technique. The author, Alan Watson, gives detailed biological and mechanical explanations of these problems. The book covers a wide array of instruments as well as performance situations, and detailed analysis as to the biological outcomes or problems of each. The traditional keyboard produces its own unique variety of physical symptoms. The most common problems associated with piano playing are tendonitis and carpal tunnel syndrome, often caused by the excessive ulnar deviation in the wrist. Without highly refined technique, it can be difficult for pianists to avoid these issues due to crossing fingers and the stretching that occurs with large or awkward voicings. One argument in support of the isomorphic keyboard design is that it is more ergonomic than a traditional keyboard. The isomorphic keyboard is not necessarily the best theoretical model of a midi controller, or even the most ergonomic. To be more specific, it is the best option to input precise notes simultaneously using finger movement on a flat plane. There are other ways for musicians to connect to their physical environment than through the exclusive use of hands and fingers. Audio to midi data conversion has been an emerging technology. The

2010 article, “An Interactive Whistle-to-Music Composing System Based on Transcription,

Variation and Chords Generation,” is about the relationship between audio data and midi data.

The authors were interested in creating an application capable of translating audio data (from someone whistling a melody) into midi data, which could then be used to transcribe the melody Earnest 10

into music notation software. Many variables needed to be considered such as pitch, timbre, natural fluctuations, or errors occurring in the whistling. This technology is a glimpse into the vast new world of possibilities opening up for acoustic musicians, and a way for them to harness the raw live energy of an acoustic performance and embellish their own timbre using actively responsive electronics. This technology also comes standard in Ableton Live 10. People have ​ ​ been turning beatboxing into digital drum beats and hummed melodies into MIDI notes for several years now. The integration of audio to midi conversion software with an isomorphic midi controller like the Push 2, can add even more dimensions to the music. ​ ​ Besides using keyboards to transmit data, there are other controller alternatives that translate spatial movements into data. Although many of these devices are more limited when producing polyphonic music, such as the keyboards discussed earlier, they offer an extreme example of utilizing multiple dimensions to create music. These controllers commonly accomplish the task of translating the raw data readings of accelerometers into midi notes and control messages. This is even more relevant because of the prevalence and accessibility of smartphones, which almost all have some sort of accelerometer. Of course, in order to turn these devices into midi controllers, a lot of musically intelligent programing must occur. Most smartphones also have a camera, a microphone, touch screen capabilities, plus auxiliary and

USB capabilities which expand their potential even more. Hypothetically, the camera, speaker, and accelerometers on the phone could be used to receive light, audio, and motion data, which through programming could be translated into MIDI data. Even tablets can be turned into mini-isomorphic midi controllers. There are pros and cons to this method of performance. Earnest 11

Although it would be more difficult to hit precise notes, with good programming, software patches can be made to filter the data into any desired group of notes.

There are several artists who have already made use of smartphones in performance.

Perhaps the most notable is Onyx Ashanti, who uses several iPods, as well as a custom made contraption that has taken the basic concept of an EWI (electronic wind instrument), and has reconstructed it to free his arms from the constraint of holding a large clarinet like instrument.

His design uses saxophone fingerings and a breath and bite sensor. He uses the motion of both hands to trigger data through the use of accelerometers. Ashanti has coined the term “Beat Jazz” as the name of his genre due to its obvious routes in improvisation and jazz (especially because of his “converted” EWI) as well as EDM. Onyx Ashanti exemplifies a musician with traditional training (in jazz) who is pushing the envelope of the genre.

There are also products in the works such as a device known as the AUUG motion, which is a physical extension of the iphone, that allows for thumbless/grip-less control of the device, which maximizes ergonomic control and frees up the fingers to play notes on the touchscreen.

The AUUG motion also comes with patches that translate motion into midi messages. Just about any kind of sensor that transmits data can be reprogrammed to trigger midi messages. One artist who takes this to an extreme is SIGMA, who uses a heartbeat monitor, a breath and bite sensors, as well as motion capture sensors all over his body. These sensors have been programmed to trigger very specific sounds. It could be argued that most of the composed aspects of music exist within the programming.

When creating electronic music, what possible advantages would a piano keyboard have to one of the other midi controllers mentioned? Many musicians were trained on the piano Earnest 12

playing a classic repertoire. Some musicians who may not have time to apply their theoretical musical knowledge to the new devices, can only visualize the music on a traditional keyboard.

Obviously quality music can be made using a piano keyboard as a MIDI controller. However, this music can be inherently limited by the physical nature of the piano keyboard. For example, many of Chopin’s piano works incorporate large leaps with the left hand from low register bass notes to mid register chords. This phenomenon can also be seen in stride and jazz piano transcriptions especially the work of piano genius Art Tatum. These large leaps require that small rests are placed between the bass notes and chords. This means that the physical nature of the piano is literally affecting the rhythm and harmonic structure of the music. The use of a piano keyboard as a MIDI controller in electronic music will ultimately result in producing a

“nostalgic” tone reminiscent of an age dominated by acoustic instruments. Because of the elegance of computer programming, complex sounds can be produced or triggered through the transmission of MIDI data using a multitude of new ergonomic controllers. For innovative electronic musicians uninterested in basking in the glory of the past, and who understand the transient and ever evolving nature of music, it is worth considering that there are significant drawbacks and limitations to using a piano style keyboard as a midi controller. Those interested in creating cutting edge music should consider using alternative MIDI controllers such as isomorphic keyboards or other devices that utilize spatial movement and other sensors.

In an increasingly electronic musical landscape, proper respect and attention is being given to electronic artists. Understanding and acknowledging the beauty and depth that can exist in ALL types of music are key to breaking free of conservative traditionalist views towards.

Although it is more than acceptable to use a classic repertoire for demonstrating musical Earnest 13

techniques, it is important to acknowledge the limitations that classical composers faced when writing music for acoustic instruments in the first place. It is also important to understand the specific limitations of acoustic controlling interfaces like the piano keyboard, and realize that they have been designed to accommodate often large acoustic sound making mechanisms.

Creative people constantly reevaluate the old or common sense ways of doing things. They love to push the envelope of what is possible. They can be obsessed with maximizing efficiency.

Unfortunately even the most creative people can often find it extremely easy to fall into the comfort of old routines without question. This is why it is important to question the legitimacy and effectiveness of the piano as a MIDI-controller in the world of electronic music, and foster a tradition of exploration by seeking out potentially more practical and powerful instrument designs. The adoption of new alternatives can explode the creative potential of music and art in the digital age.

Earnest 14

References

About This Issue. (2007). Computer Music Journal,31(4), 1-2. doi:10.1162/comj.2007.31.4.1 ​ ​

Hopkin, B., & Scoville, J. (2010). Musical instrument design: Practical information of ​ musical instrument making. Tuscon, AZ: See Sharp Press. ​

Isomorphic | Definition of isomorphic in English by Oxford Dictionaries. (n.d.). Retrieved

from https://en.oxforddictionaries.com/definition/isomorphic

Kusek, D., & Leonhard, G. (2005). The future of music: Manifesto for the digital music ​ revolution. Boston: Berklee Press. ​

Maupin, Steven & Gerhard, David & Park, Brett. (2011). Isomorphic Tessellations for ​ Musical Keyboards. Proceedings of the 8th Sound and Music Computing Conference, ​ SMC 2011.

Milne, A., Sethares, W., & Plamondon, J. (2007). Isomorphic Controllers and Dynamic

Tuning: Invariant Fingering over a Tuning Continuum. Computer Music Journal,31(4), ​ ​ 15-32. doi:10.1162/comj.2007.31.4.15

Of Patterns and Patternists; A Sonocybernetic Manifesto. Retrieved from

http://onyx-ashanti.com/

Earnest 15

Shen, H., & Lee, C. (2010). An interactive Whistle-to-Music composing system based on

transcription, variation and chords generation. Multimedia Tools and Applications,53(1), ​ ​ 253-269. doi:10.1007/s11042-010-0510-6

Snoman, R. (2011). Dance music manual. Amsterdam: Focal Press. ​ ​

Tessellation | Definition of tessellation in English by Oxford Dictionaries. (n.d.). Retrieved

from https://en.oxforddictionaries.com/definition/tessellation

Watson, A. H. (2009). The biology of musical performance and performance-related injury. ​ ​ Lanham, MD: Scarecrow Press.