[Translating and the Computer 24: proceedings of the…International Conference…21-22 November 2002, London (Aslib, 2002)] Japanese Language: An Introduction for Language Technology Professionals and Researchers Carl Kay Independent Consultant 15-22, Onoharanishi 3-chome Minoh-shi, Osaka Japan 562-0032 [email protected] David Fine Internationalization Consultant Philadelphia, PA United States [email protected] Background and Synopsis It is the authors' experience, based on a combined four decades in the translation and localization business, that most people involved in the language business lack knowledge about the basic features of the Japanese language and how it is the same or different from major Western languages. This is quite natural given the many ways that Japan and Japanese are remote presences to most people outside Japan. Many barriers make Japanese inaccessible to Westerners: a strange and complex written language (and related challenges in computer processing of text); cultural differences; limited opportunities to study the language (and the time such study requires); geographic and time zone gaps, and Japan's shallow interaction with the West compared to exchanges among Western nations. Still, even in a recession Japan remains one of the major economies of the world. Its information technology market still dwarfs that of the rest of Asia. In software localization Japanese is often a "Tier One" language meaning a target of the first round of localization aimed at the biggest markets not addressable with English product. Japan's government and companies remain major funders of research in machine translation and other language technology. Yet the above-mentioned barriers make it hard for Westerners in the language technology field to handle contact with Japanese as comfortably and effectively as they do with many other languages. The authors' goal here is to allow language professionals and researchers outside Japan to become more familiar with Japanese and to feel more "at home" when dealing with Japanese in their work. The structure of this paper is to describe four aspects of Japanese language and culture that are different from Western languages and cultures and have important impact on the language technology business. These differences are: 1. Extremely large Japanese character set, and lack of fit to the language itself 2. Extreme abundance of Japanese homonyms, lack of word spacing, and related input and sorting issues 3. Japanese structural features including S-O-V structure, placement of relative clauses, and omitted or indirect subjects 4. Japan's lack of historical contact with Western nations relative to contact among the Western nations For each of the above items, the authors will give an explanation of the origins and nature of the situation, the implications for the language technology business, and trends going forward. The emphasis is on a practical overview rather than detailed scholarly discussion. The authors also limit the discussion of Western languages mainly to English while acknowledging that at times we are missing important distinctions between major Western languages. Our emphasis is on the broader contrast of Japanese and the West rather than the finer points that could be made by those with more background. Feature #1 Extremely large Japanese character set, and lack of fit to the language itself Origins and nature of the situation: The Japanese writing system is largely borrowed from Chinese. Around the third or fourth century A.D., when Chinese civilization was quite advanced in terms of written language, government structure and philosophy, Japan was a loosely organized group of clans who had no writing system. The spread of Buddhism into Japan from China and Korea was accompanied by the introduction of the Chinese writing system which consisted of tens of thousands of characters. Most Chinese characters represent a single word. The visual depiction of some Chinese characters derives from the meaning of the word represented. Other more complex characters are built from smaller elements, which may be characters in themselves though not necessarily. Some of these smaller elements are called radicals and carry a degree of category meaning. For example, many of the words related to botany contain a common element called the grass radical, which gives some framework to the meaning of the character but no clue at all to its pronunciation. Additional elements of each character under a given radical differentiate the many individual characters in the category. In some cases, but not always, these additional elements give some indication of the pronunciation of the character. More abstract words are often represented by a sequence of two characters called a compound. When the Chinese written language came into Japan, it came packaged with many elements of Chinese Confucian culture and with thousands of new words incorporating the ideas and forms of that culture. Japanese culture even today in the 21st century remains highly shaped by this original importing of Chinese culture and writing system. Of course, many elements were adapted or dropped over time. Despite this great contribution China made to Japanese culture, in many ways the Chinese written language is not a good fit to record the Japanese language, which is quite distant linguistically from Chinese. This mismatch has contributed to the extreme complexity of the Japanese writing system in use today. One such mismatch is in verb structure. Japanese verbs are highly inflected, with many elements added onto the verb stem to indicate tense and active or passive voice. Chinese does not make these distinctions by adding on to the verb word itself. Thus, lacking a writing system, the Japanese seemingly happily borrowed the Chinese character for "eat" as the Japanese character for "eat." (The pronunciation of the two is unrelated, but in both languages the character means "eat.") In Japanese, however, other written markers were needed to add the inflections, such as present or past tense indicators, to the word "eat." Having no writing system in place, Japanese also took these indicators from Chinese as well, this time in the form of Chinese characters that happened to have approximately the same pronunciation as the inflection markers in Japanese (but having unrelated meanings in Chinese, being of interest to Japanese writers only for their pronunciation). Thus the earliest form of written Japanese consists of borrowed Chinese characters, some of them there because of the meaning borrowed from Chinese and some there because of the sound borrowed from Chinese. Reading ancient Japanese documents was probably challenging in their time and is certainly extremely so now. The Chinese characters borrowed for their meaning are called kanji ("Chinese characters") in Japanese and remain a major part of the written language. Meanwhile, over time, the Chinese characters borrowed just for their sound were simplified and evolved into a phonetic alphabet of characters called hiragana. All Japanese speech can be recorded with the hiragana character set, but in actual practice a mixture of kanji (for which the pronunciation must be memorized) and hiragana are used. Going back to the above example, the Japanese word for "ate" can be written with the kanji for "eat" pronounced "ta-be" followed by hiragana "ma-shi-ta" indicating past tense, or the word can be written entirely in hiragana: ta-be-ma- shi-ta. There are many rules and conventions about where to use kanji and where to use hiragana, which are called okurigana when used to represent the inflections of verbs and adjectives (which are similarly inflected in Japanese). Schools tend to teach children to read and write hiragana first, adding kanji of increasing complexity each year according to a fixed schedule at each grade level. An alternate set of phonetic characters called katakana further complicates the situation. Katakana characters duplicate the phonetic coverage of the hiragana character set but are used mainly for words of western derivation, to distinguish them from native Japanese words. For example, the word Ko-n-pyu-ta spells out the word computer in Japanese pronunciation. There is also a Japanese word for computer, keisanki, composed of three kanji characters, but it is rarely used. Japanese Character Sets for Electronic Processing The large and complicated Japanese character set makes electronic processing of Japanese text an order of magnitude more complex than that for Western languages. The Japan Standards Association has defined a series of standards which specify codes for Japanese characters. Some of the key Japan Industrial Standard (JIS) character sets are described below. The JIS X 0201 (Code for Information Interchange) standard defines a 7-bit character set for Roman characters, a 7-bit character set for katakana characters, and an 8-bit character set for both Roman and katakana characters. The JIS X 0201 Roman character set is very similar to the ASCII character set but the backslash has been replaced by the Yen symbol and the tilde has been replaced by the "overline". For anyone who has used Japanese DOS or Japanese Windows this explains why the path separator "\" displays as a Yen symbol. JIS X 0208 (Code of the Japanese Graphic Character Set for Information Interchange) specifies the set of graphic characters and accompanying codes used in normal written Japanese. The class of graphic characters contained in JIS X 0208 includes special characters, numeric characters, Roman characters, hiragana, katakana, Greek characters, Russian characters, kanji, and line elements. The 6,355 kanji designated in the standard are divided into two levels. The first level contains the most frequently used kanji sorted by representative reading (as will be explained in more detail later, most characters have multiple possible readings), and the second level contains less common characters arranged according to radical and stroke count (based on the number of brush or pen strokes needed to write the character on paper). Each character is uniquely identified by two 7-bit bytes.