Proceedings of the 23rd Conference of the Oriental COCOSDA November 5-7, 2020, Yangon, Myanmar Formosa Speech Recognition Challenge 2020 and Taiwanese Across Taiwan Corpus Yuan-Fu Liao Chia-Yu Chang Hak-khiam Tiun National Taipei University of Technology National Taipei University of Technology National Taitung University Taipei, Taiwan Taipei, Taiwan Taitung, Taiwan yfl[email protected] [email protected] [email protected] Huang-Lan Su Hui-Lu Khoo Jane S. Tsay National Taitung University National Taiwan Normal University National Chung Cheng University Taitung, Taiwan Taipei, Taiwan Chia-Yi Min-Hsiung, Taiwan [email protected] [email protected] [email protected] Le-Kun Tan Peter Kang Tsun-guan Thiann National Cheng Kung University National Donghwa University National Taichung University of Education Tainan, Taiwan Hualien, Taiwan Taichung, Taiwan [email protected] [email protected] [email protected] Un-Gian Iunn Jyh-Her Yang Chih-Neng Liang National Taichung University of Education Telecom. Lab., Chunghwa Telecom Automotive Research & Testing Center Taichung, Taiwan Taoyuan, Taiwan Changhua, Taiwan [email protected] [email protected] [email protected] Abstract—Taiwanese (a.k.a. Taiwanese Hokkien, Hoklo, Taigi, Mandarin as the only official language, i.e., ”國語”, banning Southern Min or Min-Nan) is an endangered language, because Taiwanese’s use in schools or formal occasions and limiting the domination of Mandarin, the number of Taiwanese speakers the hours of Taiwanese broadcast radio or TV programs. The continues to drop, especially among the youth generations. In addressing this problem, a Taiwanese speech-enabled human- consequence is that Mandarin became dominant in Taiwan and computer interface for supporting people’s daily life is essential. although, nowadays, about 70% of the population still could Therefore, a Formosa Speech in the Wild (FSW) project was use Taiwanese [3], most people, especially among the young established to collect a large-scale Taiwanese speech across Tai- generations, only have limited vocabulary and cannot speak wan (TAT) corpus to boost the development of Taiwanese speech Taiwanese fluently. recognition (TSR). A Formosa Speech Recognition Challenge 2020 (FSR-2020) was also hosted to promote the corpus as well as More specifically, a Taiwanese language television channel to evaluate the performance of state-of-the-art TSR systems. This (Taigi) of the Public Television Service (PTS) [4] was not paper briefly introduces TAT corpus and FSR-2020 challenge, established until 2019 after the Taiwan National Language presents the provided data profile, evaluation plan and reports Development Act finally gave all of the languages spoken in experimental baseline results. A subset of TAT corpus, TAT- Taiwan equal status as the country’s official national languages Vol1, is given away for free for all participants (non-commercial 全家有 license), and its corresponding Kaldi baseline recipes have been in 2018. Moreover, one popular Taigi’s programs, the ” published online. Experimental results have showed that the 智慧” (Smart Family) quiz show, which takes the fun-learning combination of TAT corpus and the baseline recipes is a good Taiwanese language puzzle competition as the main axis, in resource pack for TSR research and development. fact deeply reflects the miserable situation of Taiwanese lan- Index Terms —Taiwanese across Taiwan Corpus, Formosa guage. Because most guests and audiences of the show, even Speech Recognition Challenge 2020, Taiwanese Speech Recog- nition, Machine learning for those elderly, have somehow poor Taiwanese language ability and often have to switch back to Mandarin for smoother I. INTRODUCTION communication in the TV show. It is believed that it is not too late to save this language. Taiwanese (a.k.a. Taiwanese Hokkien, Hoklo, Taigi, South- To promote Taiwanese language, a Taiwanese speech-enabled ern Min or Min-Nan) [1] is in danger of extinction, because human-computer interface, such as Taiwanese voice command, the use of Taiwanese was discouraged by the Kuomintang spoken dialogue, speech synthesis or even automatic TV show until the 1980s [2], through measures such as assigning subtitling or translation systems, for supporting people’s daily 978-1-7281-9896-5/20/$31.00 © 2020 IEEE life is necessary. However, there is no public large-scale 65 Authorized licensed use limited to: National Chung Cheng University. Downloaded on March 09,2021 at 09:42:46 UTC from IEEE Xplore. Restrictions apply. Taiwanese speech corpus available for building a state-of-the- has many notable differences from those in mainland China. art (SOTA) Taiwanese speech recognizer, especially for those Many of the differences can be attributed to the influence from deep neural network-based approaches (usually more than 100 the languages of Formosan, Dutch and Japanese [1]. hours is required). A. Regional variations In the first phase of FSW [5] project (from 2017 to 2020), a large-scale (3,200 hours) National Education Radio (NER) There are a number of pronunciation and lexical differences broadcast Taiwanese Mandarin speech corpus [6], i.e. NER- between the Taiwanese variants. Some scholars have divided Trs-Vol1∼17 and NER-Pro-Vol1∼4, had been transcribed and Taiwanese into five subdialects [11] based on geographic publicly released. By the help of this large NER corpus, a region (as shown in Fig. 1) including: SOTA Mandarin speech recognizer had been built and many • hai-kh´ au´ (海口腔): west coast, formerly referred to as Mandarin speech-enabled applications, especially a real-time Quanzhou dialect (represented by the Lukang accent) subtitle generation system for Taiwan Centers for Disease • phian-hai´ (偏海腔): coastal (represented by the Nanliao Control (CECC) COVID-19 press conference [7] had been (南寮) accent) successfully deployed recently. • lai-po(¯ 內埔腔): western inner plain, mountain regions, Being inspired by previous success, in the second phase based on the Zhangzhou dialect (represented by the Yilan of FSW project (from 2019 to 2011), we switched our focus accent) to collect a large-scale (more than 300 hours) Taiwanese • phian-lai¯ (偏內腔): interior (represented by the Taibao across Taiwan (TAT) corpus. The results will be used as accent) an infrastructure to boost the research and development of • thong-hengˆ (通行腔): common accents (represented by Taiwanese speech recognition (TSR). However, transcription the Taipei (spec. Datong) accent in the north and the of Taiwanese speech is more difficult than Mandarin due the Tainan accent in the south) following issues: • too many regional variations • no standard writing system • only few people can speak Taiwanese fluently • only few well-trained experts are capable of phonetically transcribing a given Taiwanese speech Therefore, instead of transcribing spontaneous Taiwanese speech, an alternative strategy was chosen to alleviate those difficulties: • recruit native Taiwanese speakers across Taiwan to cover regional variations • adapt the official Taiwanese Romanization system [8] proposed by Taiwan Ministry of Education (MOE) as the writing standard. • record reading speech with prepared prompt sheets in- Fig. 1. Distribution of Taiwanese sub-dialects around Taiwan. stead of transcribing spontaneous speech by linguists The official MOE Taiwanese dictionary [12] further speci- Currently, the first part (100 hours) of TAT corpus, TAT- fies these variations to a resolution of eight regions on Taiwan, ∼ Vol1 2, has been finished. Therefore, we would like to host in addition to Kinmen and Penghu. a FSR-2020 challenge to promote the corpus as well as to evaluate the performance of potential state-of-the-art TSR B. Scripts and orthographies systems. So, we are now calling for and welcome participants Taiwanese does not have a strong written tradition. Until the from both academic and industrial sectors to FSR-2020 [9]. late 19th century, Taiwanese speakers wrote mostly in classical In the following sections, TAT corpus and the FSR-2020 Chinese (Han-j` ¯ı, 漢字). However, there are various problems challenge will be briefly introduced. Especially, the statistics relating to the use of Chinese characters to write Taiwanese, of TAT-Vol1∼2 will be given and the details of the FSR-2020 since many morphemes (around 15% of running text) are not including the provided data profile, evaluation plan and the definitively associated with a particular character. experimental baseline results will be reported. It is believed Therefore, people began to use various different methods the combination of TAT corpus and baseline recipe [10] is a to solve this issue. Such as using either a Latin-based script good resource pack for further TSR research and development. or through the use of a Chinese character chosen phonetically with no relation to the original word via meaning. II. BACKGROUND Among many systems of writing Taiwanese using Latin Taiwanese is a branched-off variety of Southern Min di- characters, the most used system is called peh-oe-j¯ ¯ı (POJ, 白 alects. However, due to history and geographical separation, 話字) developed by Western missionaries in the 19th century. Taiwanese developed independently from those in Fujian and POJ can also be used along with Chinese characters in a 66 Authorized licensed use limited to: National Chung Cheng University. Downloaded on March 09,2021 at 09:42:46 UTC from IEEE Xplore. Restrictions apply. mixed script called Han-l` o(ˆ 漢羅), where words specific to Taiwanese are written in POJ, and words with associated characters written in Han Characters. Recently, MOE officially promoted the Taiwanese Roman- ization System (Tai-lˆ o,ˆ 台羅) to standardize the writing system in 2006. Furthermore, in 2009, MOE formulated and released a list of Taiwanese Southern Min Recommended Characters, in total 700 Han characters, for the use of writing uniquely Taiwanese words in the Han-j` ¯ı or mixed Han-l` o(ˆ 漢 羅) approaches. III. DATA COLLECTION The aim is to construct a proper speech corpus for building a high-performance automatic TSR system. To avoid the labor- Fig. 2. The daily conversation session of a typical prompt sheet used during intensive Taiwanese speech transcription efforts, in the phase II recording.