Proceedings of the Workshop on Speech-to-Speech Translation: Algorithms and Systems, Philadelphia, July 2002, pp. 85-92. Association for Computational Linguistics. Automatic Interpretation System Integrating Free-style Sentence Translation and Parallel Text Based Translation Takahiro Ikeda Shinichi Ando Kenji Satoh Akitoshi Okumura Takao Watanabe Multimedia Res. Labs. NEC Labs. 4-1-1 Miyazaki, Miyamae-ku, Kawasaki, Kanagawa 216 [email protected], [email protected], [email protected], [email protected], [email protected] Abstract being produced by the total system even for a lim- ited domain. Clearly, we need ways to complement This paper proposes an automatic in- speech-to-speech translation systems that cannot re- terpretation system that integrates free- liably produce a correct result. style sentence translation and parallel text Although some robust methods that make the er- based translation. Free-style sentence roneous results of other components acceptable have translation accepts natural language sen- been proposed (Yumi et al., 1997; Furuse et al., tences and translates them by machine 1998), there is no guarantee that the final output translation. Parallel text based translation from a system will be appropriate even with these provides a proper translation for a sen- methods. To deal with this problem, we have taken a tence in the parallel text by referring to a more practical approach to developing an automatic corresponding translation of the sentence interpretation system where the user can obtain a and supplements free-style sentence trans- correct result instead of having to apply additional lation. We developed a prototype of an au- operations and judgment. tomatic interpretation system for Japanese In actual use of a speech-to-speech translation overseas travelers with parallel text based system, an error in the speech-recognition or speech- translation using 9206 parallel bilingual synthesis components is not a large problem if the sentences prepared in task-oriented man- system has a screen that displays each result. The ner. Evaluation results show that the par- user of the system can correct errors in the recogni- allel text based translation covers 72% of tion result on the screen, and can communicate by typical utterances for overseas travel and showing the other person the translated sentence on the user can easily find an appropriate sen- the screen. tence from a natural utterance for 64% of On the other hand, an error in the machine- typical traveler’s tasks. This indicates that translation component is critical because a user who the user can benefit from reliable transla- is not familiar with the target language is unlikely tion based on parallel text for fundamental to notice the error in some cases. When a nonsensi- utterances necessary for overseas travel. cal sentence is generated by machine translation, the user may realize that the listener does not understand 1 Introduction the translated sentence. However, when a plausible sentence that means something different from the in- A speech-to-speech translation system must inte- tended meaning is generated by the machine trans- grate at least three components — speech recogni- lation, the user may incorrectly assume that the ut- tion, machine translation, and speech synthesis. In terance was properly communicated. Consequently, practice, each component does not always output the user can seldom be sure that the listener cor- the correct result for various inputs, and an error rectly understood the intended meaning when using in one component often leads to an incorrect result a speech-to-speech translation system. A conversa- tion could continue for some time before it became 2 The Integration Model apparent that the two sides misunderstood what the 2.1 User Interface other was saying. Moreover, if the user realizes that there is an er- Although parallel text based translation provides a ror in the machine translation, correcting it will be correct result, the registered parallel bilingual sen- difficult. Without knowing the source of the error, tences cannot cover all possible utterances by the the user cannot modify the input to obtain a correct user in the target domain. Free-style sentence trans- result. lation, on the contrary, accepts free-style input sen- These error problems severely limit the usability tences but provides no guarantee as to the quality of of speech-to-speech translation. results. In this paper, we propose an automatic interpreta- For many routine situations, users will clearly tion system that integrates free-style sentence trans- benefit from using parallel text based translation. lation and parallel text based translation. In this sys- In such cases, the system will probably include a tem, free-style sentence translation accepts natural sentence that totally or partially fits what they want language sentences and translates them by machine to say. To ensure high translation reliability, users translation without guaranteeing the quality of the should use free-style sentence translation only for translation. On the other hand, parallel text based utterances not covered by the registered sentences. translation uses parallel bilingual sentences regis- However, users usually will not know what sen- tered in the system and translates a registered sen- tences are registered in the system and will have to tence by referring to the corresponding translation. search for an appropriate sentence before they can Although this translation process limits the input to use parallel text based translation. In some cases, the registered sentences, it is a robust means of han- user will be forced to use free-style sentence trans- dling input with recognition errors and consistently lation if unable to find an appropriate sentence. provides a correct translation. We integrated these A seamless user interface that allows the user to two types of translation to realize a robust transla- easily switch between free-style sentence transla- tion system where the two types of translation com- tion and parallel text based translation is therefore pensate for the shortcomings of each other. needed in a system integrating these two forms of For appropriate integration of free-style sentence translation. Two conditions in particular had to be translation and parallel text based translation, we met to make the system easy to use. had to consider three main points. 1. The user should be able to use an input sen- tence seamlessly as both a source sentence for 1. User interface: how best to present the two free-style sentence translation and a key sen- functions to the user? tence for registered sentence retrieval. 2. Content of registered sentences: How many ut- 2. The user should be able to use each sentence in- terances should be covered by registered sen- cluded in the results of the registered sentence tences? retrieval and the input sentence as a source sentence for translation. (The former would 3. Retrieval system: What methods of searching be used for parallel text based translation, and among the registered sentences should be pro- the latter would be used for free-style sentence vided to the user? translation.) In this paper, we discuss these three points with 2.2 Content of Registered Sentences respect to a translation system for Japanese travelers Registered sentences must cover the utterances nec- in the overseas travel domain. We construct a model essary for accomplishing typical tasks in the target of the integration of free-style sentence translation domain to provide correct translation for minimal and parallel text based translation in Section 2. We communication. In a translation system for overseas describe a prototype system based on the model in travelers, some typical tasks are changing money, Section 3 and evaluate it in Section 4. Related work checking in at a hotel, and ordering at a restaurant. on translation systems utilizing parallel text are dis- We adopted a three-tier model that consists of cussed in Section 5, and we conclude in Section 6. scenes, tasks, and subtasks to prepare a sufficient set Table 1: Examples of scenes, tasks, subtasks, and templates of sentences Scene Task Subtask Template of sentence Hotel Check-in Checking in I’d like to check in, please. Hotel Check-in Requesting a type of room I’d like a room with the ocean view. Restaurant Order Requesting cooking time for your steak Medium, please. Restaurant Order Asking what they recommend What do you recommend for appetizers? of necessary sentences to be registered in the sys- for the respondent to choose from. The system keeps tem. A scene comprised a place or situation that typical responses in parallel bilingual form for each corresponds to where a traveler is likely to be (e.g., a registered sentence that the traveler can use and dis- hotel) and a problem that could arise. We made a list plays these as candidate responses when the traveler of typical travelers’ tasks that would be necessary in uses the sentence. The system then shows the trav- various travel scenes, divided each task into smaller eler the translation of the response selected by the primitive tasks (subtasks), and assigned a sentence respondent. template to each subtask based on the model. This approach enables travelers to obtain a reli- In general, more than one round of conversation able response and also enables respondents to easily is necessary to accomplish each task. We assumed select an appropriate response. that a task would consist of smaller subtasks, each of which would correspond to one round of conver- 2.3 Retrieval System sation that consisted simply of an utterance from a The retrieval system to search for a registered sen- traveler to a respondent and a response from the re- tence that we use is based on a combination of three spondent to the traveler. For example, the task of conditions — the natural language sentence, scene, checking in to a hotel consists of subtasks such as and action.