Construction and Verification of the Scale Detection Method for Traditional Japanese Music – a Method Based on Pitch Sequence of Musical Scales –

International Journal of Affective Engineering Vol.12 No.2 pp.309-315 (2013)

Special Issue on KEER 2012 ORIGINAL ARTICLE

Construction and Verification of the Scale Detection Method for Traditional Japanese Music – A Method Based on Pitch Sequence of Musical Scales –

Akihiro KAWASE

Department of Corpus Studies, National Institute for Japanese Language and Linguistics, 10-2 Midori-cho, Tachikawa City, Tokyo 190-8561, Japan

Abstract: In this study, we propose a method for automatically detecting musical scales from Japanese musical pieces. A scale is a series of musical notes in ascending or descending order, which is an important element for describing the tonal system (Tonesystem) and capturing the characteristics of the music. The study of scale theory has a long history. Many scale theories for Japanese music have been designed up until this point. Out of these, we chose to formulate a scale detection method based on Seiichi Tokawa’s scale theories for traditional Japanese music, because Tokawa’s scale theories provide a versatile system that covers various conventional scale theories. Since Tokawa did not describe any of his scale detection procedures in detail, we started by analyzing his theories and understanding their characteristics. Based on the findings, we constructed the scale detection method and implemented it in the Java Runtime Environment. Specifically, we sampled 1,794 works from the Nihon Min-yo Taikan (Anthology of Japanese Folk Songs, 1944-1993), and performed the method. We compared the detection results with traditional research results in order to verify the detection method. If the various scales of Japanese music can be automatically detected, it will facilitate the work of specifying scales, which promotes the humanities analysis of Japanese music. Keywords: Traditional Japanese music, Theory of Musical Scales, Tonesystem

composers, musicians, song pieces, or how the audience 1. INTRODUCTION interprets musical experiences, in terms of the art music of the common practice periods identified as Baroque, 1.1 Purpose of the study Classical and Romantic in Western Europe. However, The main purpose of this study is to partially express the there is also an undeniable sense that many aspects of concept of Japanese music by estimating the structures musical structure may play an important role in non- from folk songs. Western music as well, and research achievements from A scale is a series of musical notes in ascending or many practical fields have not yet utilized or led to stud- descending order, which is an important element for ies of ethnomusicology [6]. Thus, the question “How did describing the structure (the tonal system, Tonesystem) non-Western cultures conceptualize their own music?” and capturing the characteristics of the music [1]. remains unsolved. In addition, methods in the field of A fundamental aspect of musical analysis and folk song comparative musicology integrate fieldwork in cultural research is to clarify the scale in a musical piece [2]. anthropology and document investigation in historical Many scale detection models have been developed. For science based on a humanities approach [7]. In this instance, a method for tracing chord progressions of respect, there have been many case studies about non- Western music was developed based on Noam Chomsky’s Western music from a musicological point of view, but generative grammar [3-4], and discriminating tonality of those studies were conducted with a small amount of Japanese music was developed based on Japanese yona data, and have rarely been reconfirmed by objective nuki musical scales (extracting the fourth and the analysis utilizing large amounts of data for computa- seventh note from heptatonic scales) [5]. However, a tional analysis. detection model that encompasses the essence of tradi- If the various scales in Japanese music can be tional Japanese music has not yet been developed. automatically detected, it will facilitate the work for Traditionally, in the fields of cognitive science and specifying scales, which people have traditionally done music analysis that use computer-based technologies, by hand, and will promote the humanities analysis of researchers have mainly been concerned with specific Japanese music. We will start by implementing the scale

detection method [8], and then verify its validity by using Chinese music theory and Gong in Japanese music theory it on the music corpora of Japanese folk songs. represent the same pitch height, although the terms are different. 1.2 Organization Although the Japanese adopted many musical terms This paper is organized as follows. In Section 2, we from the original Chinese, these terms were pronounced will first look at the major scale theories in Japanese differently in Japan. For descriptive purposes, here we music to summarize important concepts and terms related will use {C, D, E, G, A (, C)} instead of {Gong, Shang, to scale detection method. Based on this understanding, Jue, Zhi, Yu (, Gong)} according to the equivalent of Section 3 outlines the basic principles of Seiichi Tokawa’s moving the C note around the 12 pitches of the Western scale theories. Section 4 gives an outline of our scale scale. detection method with an experiment of how a scale of the national anthem of Japan can be detected. Section 5 2.2 In (Yin) scale and Yo (Yang) scale gives an overview of the musical data used for verifying In the Meiji period (1868-1912), in his book Zokugaku the detection method. Then, in Section 6, we implement Senritsu Ko (On the Melodies of Japanese Vernacular the detection method and apply to the musical data. We Music) [11], a musicologist Rokushiro Uehara classified compare the detection results with traditional research the native songs into two types: the In scale (In mode results, and discuss the validity of the detection method. or Miyako-bushi scale) and the Yo scale (Yo mode or In Section 7, we conclude the paper by summarizing the Inaka-bushi scale). Uehara’s greatest contribution is that whole study and describing future topics. he approximated the ancient Chinese pitches with the 12 Western chromatic semitones, which made it possible to 2. SCALES THEORY IN JAPANESE MUSIC express the melodies of Japanese music on Western staff notation. However, there were many difficulties in In the 12th century, music and musical instruments, as explaining all Japanese music using the two scales, the In well as Buddhism and Confucianism, were originally scale and the Yo scale. Many scholars, including Shuji introduced from China, which developed independently Isawa, the first Chief of the Education Bureau of Japan, into Japanese music. Since there was a strong tendency brought counterarguments during the last half of the to focus on individual songs or playing styles instead Meiji period to the first part of the Showa period of theoretical aspects, Japanese music did not have a (1926-1989). theoretical system until Western music was imported as a part of the Europeanization policy during the Meiji period 2.3 Koizumi’s tetrachord theory (1868-1912) [9]. Influenced by the methods of Western comparative musicology, Fumio Koizumi conceived of a scale based 2.1 Ritsu scale and Ryo scale on the interval of a perfect fourth, and has developed his It is considered that the musical scales brought from tetrachord theory [12]. As shown in Figure 1, a tetrachord China during the time from the Nara period (710-794) to is a unit consisting of two stabled outlining tones called the Heian period (794-1185), were roughly classified into kaku-on (nuclear tones or Kernton), and one unstable two groups: the Ritsu scale and the Ryo scale. In the scale intermediate tone, four different types of tetrachords can theory of ancient China, {Gong, Shang, Jue, Zhi, Yu be formed: the Min-yo, the Miyako-bushi, the Ritsu, and (, Gong)} were given to each pitch of the scale as a unique the Ryu-kyu. name. Also, any of these five pitches can be cyclically Koizumi suggested that combinations of two identical shifted as a tonic (first pitch of a scale), which will gener- tetrachords with an intermediate tone generate a one of ate five different scales. Although the name of the tonic four Japanese scales named after each tetrachord: the was not replaced in China, the tonic was always read as Min-yo scale, the Miyako-bushi scale, the Ritsu scale, and Gong in Japan. It became customary to allocate Gong to any tonic in Japanese scale theory [10]. For example, in a scale in which Shang is placed on the tonic, the names of each pitch would become {Shang, Jue, Zhi, Yu, Gong (, Shang)} in Chinese music theory, while {Gong, Shang, Jue, Zhi, Yu (, Gong)} is universally applied in Japanese music theory. In this case, Shang in Figure 1: Koizumi’s four basic tetrachords

310 International Journal of Affective Engineering Vol.12 No.2

Construction and Verification of the Scale Detection Method for Traditional Japanese Music the Ryu-kyu scale [12]. In addition, combinations of Miyako-bushi scale, Ritsu scale, and Ryu-kyu scale corre- different tetrachords (conjunct and disjunct) can also be spond to Tokawa’s Yo-type A-mode, In-type E-mode, consisted to generate a new scale [13]. Yo-type G-mode, and Ryu-kyu-type C-mode, respectively. Since Tokawa’s theory is not based on the tetrachord 3. TOKAWA’S SCALE THEORIES theory, there is criticism that Tokawa’s classification cannot capture the fundamental features of Japanese 3.1 Four major types of Tokawa’s musical scales music. However, Tokawa’s theory still provides a versatile Seiichi Tokawa, a Japanese music theorist, criticized system which covers various conventional scale theories that the problem with scale theory in Japanese music is under the present circumstances. rooted in the ready-made concept that a musical scale is conventionally specified as one mode per type, a classifi- 3.2 Characteristics of Tokawa’s theory cation methodology that is found in today’s sheet music Since Tokawa has not described any procedures for [14-15]. Tokawa classified the scales into four major scale detection in detail, we will discuss the features of types: Yo, In, Kon-go, and Ryu-kyu. Basically, each scale Tokawa’s theories, using a diatonic scale, a fundamental has five pitches, so we can generate five scales (or modes) scale in Western music, as an example, which is important per type by allocating the tonic, respectively. in divining the scale detection method. Table 1 is a list of Tokawa’s musical scales, which is a If we take a standard C-rooted diatonic scale for exam- transcript from Nihon No Onkai wo Saguru (Exploring the ple, this scale is composed of a sequence of the following Japanese Musical Scale) [14] with partial modifications to seven notes: {C, D, E, F, G, A, B (, C)}. As we cyclically pitch names using the alphabetic letters ABCDEFG shift the starting note of the scale, we can obtain seven instead of Chinese and Japanese characters. The left-hand different new musical scales (or modes). Table 2 is a list side presents the scales whose pitch names are allocated in of seven diatonic scales modeled after Tokawa’s theory. Tokawa’s arrangement (cyclically shifted order), and the On the left-hand side of Table 2, in the same manner as in right-hand side has the same scales as the left-hand side, Table 1, scales are arranged so that any of these seven but the only difference is that all tonics are lined up to A pitches can be cyclically shifted as a tonic. The difference for comparison purposes. In addition, these pitch names in each mode in Tokawa’s theory can be theoretically are described in relative pitch, and since the chromatic compared from the perspective of pitch interval. This is scale has 12 semitones (Kin or Cho in Japanese), we can obvious from the tone sequence of each scale shown on derive 12 modes from a single mode by choosing a Kin the right-hand side of Table 1, which has the same scales for the tonic. Thus, with equal temperament, Tokawa’s as the left-hand side arranged with A-rooted scales (all theory gives us 240 scales (4 types × 5 modes × 12 Kin) scales starting from A). The names of the notes described in traditional Japanese music. in each scale (or mode) also are strictly relative pitch According to this idea, Uehara’s In scale and Yo scale here, and as we choose a single arbitrary semitone for Kin correspond to Tokawa’s In-type E-mode and Yo-type G- from a range of octaves, we can generate 12 variations mode, respectively. Similarly, Koizumi’s Min-yo scale, per mode having different tonic. C-mode is the major diatonic scale (C major) and Table 1:List of Tokawa’s musical scales for traditional A-mode is the minor diatonic scale (A minor) that were Japanese music used almost solely and exclusively in Western music since the 17th century. For example, if the root note of C-mode is set to G, a scale will become G major {G, A, B, C, D, E, F (, G)}, and if the root note of A-mode is set to E, this scale will become E minor {E, F , G, A, B, C, D (, E)}. If we strictly express these two scales with the concept of Kin, type, and mode as Tokawa specified, the G major scale will become the none-type G-Kin C-mode and the E-minor scale will become the none- type E-Kin A-mode, respectively. Here, unlike traditional Japanese music, the item “type” is excluded since the concept of “type” does not exist in Western music. Thus, even if the element of each mode shown on the left-hand

311 International Journal of Affective Engineering Vol.12 No.2

side of Table 2 are just rearrangements of the first row 4. SCALE DETECTION METHOD {C, D, E, F, G, A, B (, C)}, the characteristics differ. If the tonic is unified to a single letter like the right-hand 4.1 Procedure in details side of Table 2, the difference will appear prominently. This section provides an overview of our proposed Therefore, theoretically, the slight difference in each method of distinguishing various scales from Japanese mode in Tokawa’s theory can be understood by compar- folk song pieces. Under the above circumstances, the ing from a pitch interval (difference in pitch height) scale detection method for Japanese music that we point of view. proposed based on this perspective on pitch interval is as For example, denoted by the number of semitones, the follows: pitch interval of C-mode of the first row of Table 2 {C, D, STEP 1: Count the number of pitch heights and their E, F, G, A, B (, C)} will be set to {2, 2, 1, 2, 2, 2, 1}, temporal duration from a song piece. If there are five or D-mode {2, 1, 2, 2, 2, 1, 2}, E-mode {1, 2, 2, 2, 1, 2, 2}, more pitch heights, extract the first five with the longest F-mode {2, 2, 2, 1, 2, 2, 1}, G-mode {2, 2, 1, 2, 2, 1, 2}, duration, and then set the absolute pitch height with the A-mode {2, 1, 2, 2, 1, 2, 2}, B-mode {1, 2, 2, 1, 2, 2, 2}. longest duration (mode in statistics) as the Kin. However, The main point of this example is that the sequence of if there are less than five pitch heights, stop and do not numbers in each mode is nothing but a rotated version of detect the scale. C-mode, shown in the first row. As mentioned above, we STEP 2: Sort the top five with the longest durations in focus on the pitch intervals instead of pitch heights of a alphabetical order. Denote each pitch using an integer- scale. Table 3 shows the pitch interval sequence written valued function F( • ) corresponding to the following, and out from each mode of Japanese music in Table 1. generate a pitch height vector P with six integer values, Accordingly, we construct a scale detection method tracking the value 12 onto the last element: based on Tokawa’s scale theory, and manipulate it in the F(A)=0, F(A or B )=1, F(B)=2, F(C)=3, Java Runtime Environment. F(C or D )=4, F(D)=5, F(D or E )=6, F(E)=7, F(F)=8, F(F or G )=9, F(G)=10, F(G or A )=11. Table 2: Basic musical scales for Western music STEP 3: Generate a pitch interval vector I by taking the difference between the elements that lie next to each other in vector P. Determine the type of the scale from the elements in vector I that match the pitch interval arrangement in Table 3. STEP 4: Identify the n-th location of Kin from the head of Table 3: List of Japanese musical scales denoted by the element in vector P, and create a pitch interval vector number of semitones Ic by rearranging the elements (cyclically shifted) from the n-th integer value of vector I. Determine the mode of

the scale from where Ic matches pitch interval in Table 3. STEP 5: The scale of a musical piece is determined by combining the estimated Kin, type, and mode.

4.2 Experiment with the national anthem of Japan We will implement the scale detection method proposed in the previous section and apply it to a musical piece for validation. According to Tokawa, the melody of Kimigayo, the national anthem of Japan (Figure 2) is classified as Yo-type D-Kin D-mode [16]. Based on the

Figure 2: Score of Kimigayo, the national anthem of Japan

312 International Journal of Affective Engineering Vol.12 No.2

Construction and Verification of the Scale Detection Method for Traditional Japanese Music proposal technique, we will identify the scale of Kimigayo by one and put them in order. Another reason is because step by step and see how it corresponds exactly to his although we need to depend on recording data in the pres- explanation. ent stage, the audio materials were not recorded or STEP 1: By counting the existing pitch heights and their archived under unified conditions. Thus, we need to temporal duration, we see that the pitches in the melody declare beforehand that we will conduct the analysis while of Kimigayo are {C, D, E, F, G, A, B}, and the first five recognizing certain limits of the data in the current stage. longest durations are {D, G, A, E, C}. Therefore, based on the pitch height with the longest duration, the Kin is 5.2 Statistics estimated as D. In the corpus of folk songs, we sampled the five largest STEP 2: By sorting the top five pitches with the longest song genres, which consist of 202,246 tones from 1,794 durations in alphabetical order, we get {A, C, D, E, G}, Japanese folk song pieces. Figure 3 and Figure 4 show thus, the function F( • ) returns the pitch height vector P : the distribution of the occurrence of notes and the tempo- (0, 3, 5, 7, 10, 12). ral duration (i.e., the total amount of time from “note on” STEP 3: By subtracting the elements that lie next to each to “note off”) for the respective song pieces. However, other in vector P, a pitch interval vector I: (3, 2, 2, 3, 2) here, a quarter note on a musical score is handled as 48 can be generated. Since the arrangement of the elements MIDI ticks. The mean occurrence and length were 112.78 of vector I matches the Yo-type in Table 3, this song piece notes/song and 6485.67 ticks/song, and the standard is estimated as a Yo-type. deviation for both distributions were 85.84 and 5168.06, STEP 4: Since the Kin is located in the n=3rd element in respectively. The coefficient of variation (CV), which is a vector P, by rearranging the elements in the pitch interval normalized way to measure relative variations, is defined vector I from n=3rd position, we can earn a new vector Ic : as the ratio of the standard deviation to the mean. The

(2, 3, 2, 3, 2). Since this vector Ic corresponds to the scores were 7.61 and 7.97, respectively. This indicates Yo-type D-mode in Table 3, we may estimate the mode that there is almost no difference in the variations in both as D. distributions. STEP 5: Accordingly, by combining the estimated Kin, type, and mode, the scale of Kimigayo is estimated as Yo-type D-Kin D-mode, which is exactly what Tokawa has determined in his description.

5. OVERVIEW OF DATA

5.1 Corpus of Japanese folk songs To verify the scale detection method, we use the music corpora built from the Nihon Min-yo Taikan (Anthology of Japanese Folk Songs, 1944-1993), an authoritative work of catalogued texts for Japanese folk songs. Many books of Japanese folk songs have been published, for Figure 3: Distribution of the occurrence of notes per song different objectives and applications. However, the reason why we chose the Nihon Min-yo Taikan out of a large amount of music books is that it has the definitive data that suit present condition, in terms of both quantitative and qualitative aspects [17]. Therefore, it will allow us to prepare a wealth of high-quality data when we conduct scientific analysis on Japanese folk songs. In the previous study [18], we used the same corpus in order to extract pitch transition patterns within pieces of Japanese folk songs, and to make regional classifications. In this research, the main reason for using secondary score data instead of audio materials is because it is impossible to listen to a huge number of folk songs one Figure 4: Distribution of the temporal duration per song

313 International Journal of Affective Engineering Vol.12 No.2

6. DETECTION RESULTS songs (78.88%) for the Yo type, 43 songs (17.13%) for the In type, 7 songs (2.79%) for the Kon-go type, and 3 We must take the following points into consideration as songs (1.20%) for the Ryu-kyu type. If we compare the pre-treatment items that need to be done before applying classification results which Tokawa drew from the 251 the proposal method. While Tokawa used E and G for the songs (Figure 5a) with our results obtained from the scale tonic for In scale and Yo scale (scales shown in Table 1), detection method and 1,794 song pieces (Figure 5b), it respectively, the scores included in the Nihon Min-yo turns out that the distributions of the two sets of results Taikan were taken down with relative pitch and trans- almost match. This confirms Tokawa’s hypothesis that posed with either three-flat key signatures (C minor key most Japanese music is the Yo type, only a small amount or E-flat major key) or no sharps/flats (A minor key or C of the music is the In type, and the Ryu-kyu type exists major key). These are based on the In scale and the Yo very rarely, and provides quantitative data to support it. scale with the tonic unified in G. Thus, we need to bring the tonic G back to E for scores described in In scale by 7.2 Summary and future direction transposing all pitches down a minor third prior to all In this research, we applied a scale detection method procedures. based on Seiichi Tokawa’s scale theories to musical The scale detection method was implemented in Java corpora of Japanese folk songs. Since the results match and performed on 1,794 musical pieces of the music Tokawa’s hypothesis obtained from his manual recount, corpora. The classified results have been arranged in we were able to verify that our scale detection method is order of frequency for every type (shown in Table 4). As valid. However, with this method, it was not possible to shown in Table 4, the scales of 1,794 Japanese folk songs detect a specific scale for 7.30% of the corpus. were mainly classified into three types. The Yo type was For example, the melody of Sensu-odori (folding fans the overwhelming majority with 1,381 songs (76.98%), dance) from Kagoshima Prefecture, which was one whose the In type had 233 songs (12.99%), the Kon-go type had scale we could not detect, contains seven pitches, and 49 songs (2.73%), and the Ryu-kyu type had none. 131 seems to be modulating from one scale to another in the song pieces (7.30%) could not be judged using this middle of the song. The task of detecting the scale of a method. Of this total, 95 song pieces contained six or song that has two or more scales in one melody remains a more pitches, 5 songs contained five pitches, and 31 topic for the future. Specifically, one way to handle this contained four pitches or less. would be to introduce the concept of transition entropy [5]. Furthermore, this research revealed that if the melody 7. CONCLUSIONS of a song contains less than five pitches, a scale cannot always be uniquely determined using Tokawa’s scale 7.1 Comparison with Tokawa’s survay theories. According to Tokawa’s examination [15], 251 songs For melodies like this, there are two possible directions: included in Nihon Min-yo Shu (Collection of Japanese we could estimate scale candidates applicable to Tokawa’s Folk Songs) [19] and Warabe-uta (Traditional Children’s classification system, or we could plan a new classifica- Songs of Japan) [20] were classified as follows: 198 tion index based on the position that a scale does not

Table 4: Scale detection results for 1,794 Japanese folk song pieces

Figure 5: Comparison of the classification results using pie charts

314 International Journal of Affective Engineering Vol.12 No.2

Construction and Verification of the Scale Detection Method for Traditional Japanese Music exist. Regardless of which one we choose, we will resolve 11. Rokushiro Uehara; On the Melodies of Japanese the scale detection method and assemble a database of Vernacular Music, Iwanami Paperback Library Japanese folk songs to support musical information, (1896, 1998). including scale information, historical information, and 12. Fumio Koizumi; Studies on Traditional Japanese geographic information, in order to promote folk song Music 1, Ongaku no tomo sha (1958). research from an engineering perspective. 13. Fumio Koizumi; Sounds in Japan, Shodo sha (1977). 14. Seiichi Tokawa; Exploring the Japanese Musical Scale, Ongaku no tomo sha (1990). REFERENCES 15. Seiichi Tokawa; Mode Theory, Shun jyu sha (2010). 1. The Society for Research in Asian Music; The Series 16. Seiichi Tokawa; Investigation of the National Anthem of Research in Asian Music 9, Ongaku no tomo sha of Japan, Shun jyu sha (2007). (1982). 17. Tomiko Kojima; Essay Towards Regionality Research 2. Hiroaki Katsura; The Notes on the Folk Songs in of Japanese Folk Songs 2, Journal of the Society for Akita, Memoirs of the Faculty of Education, Akita Japanese Folk Music, 12, pp.2-12 (1992). University. Education Science, 51, pp.93-99 (1997). 18. Akihiro Kawase and Akifumi Tokosumi; Regional 3. Terry Winograd; Linguistics and the Computer Classification of Traditional Japanese Folk Songs, Analysis of Tonal Harmony, Journal of Music Theory, Kansei Engineering International, 10(1), pp.19-27 12, pp.2-49 (1968). (2010). 4. Fred Lerdahl and Ray Jackendoff; A Generative 19. Kasho Machida and Kenji Asano; Collection of Theory of Tonal Music, MIT Press (1983, 1985). Japanese Folk Songs, Iwanami Paperback Library 5. Kohichi Akiyama, Minoru Matsuda and Minoru (1960). Nakano; The Discrimination of Tonality in Japanese 20. Kasho Machida and Kenji Asano; Traditional Children’s Popular Songs, Journal of the Acoustical Society of Songs of Japan, Iwanami Paperback Library (1962). Japan, 44(11), pp.809-814 (1988). 6. Alan P. Merriam; Anthropology of Music, North- western University Press (1964, 1980). 7. Tomoaki Fujii; Before ‘Music’, Japan Broadcasting Corporation (1978). Akihiro KAWASE 8. Akihiro Kawase and Akifumi Tokosumi; Scale Akihiro Kawase is a researcher at the Department Detection for Japanese Music Focused on the Pitch of Corpus Studies, National Institute for Japanese Sequence of Musical Scales, In Proc. 6th Spring Language and Linguistics, Japan. He received Ph.D. in cognitive science from the Graduate Conference of JSKE 2011, 21C-07 (2011). School of Decision Science and Technology, Tokyo 9. Yingshi Chen; Theory and Notation in China, In Institute of Technology, Japan in 2011. His Provin (Ed.), The Garland Encyclopedia of World researches are related to human perception of music. He is engaged in Music Vol.7, Routledge, pp.115-126 (2002). extracting musical concepts from both musical data and catalogued 10. Haruko Komoda and Mihoko Nogawa; Theory and texts for music criticism using statistical techniques and natural language processing techniques. He is also a member of the Society for Notation in Japan, In Provin (Ed.) The Garland Japanese Folk Music (SJFM), as well as a member of the organization’s Encyclopedia of World Music Vol.7, Routledge, Board of Directors since 2013. He won the best presentation award from pp.565-584 (2002). the Japan Society of Kansei Engineering in 2012.

315