On Beyond TM: When the Translator Leads the Design of a Translation Support Framework Reginald L. Hobbs Clare R. Voss Jamal Laoudi Army Research Laboratory Army Research Laboratory ARTI,Inc 2800 Powder Mill Road 2800 Powder Mill Road 1555 King Street Suite 400 Adelphi, MD 20783 Adelphi, MD 20783 Alexandria, VA 22314 [email protected] [email protected] [email protected] Abstract 1 Introduction Commercial off-the-shelf machine translation The Army Research Laboratory conducts experi- engines and translation support tools, such as ments in enhancing, constructing, and evaluating translation memory (TM), have been devel- MT engines and resources. The evaluation side of oped primarily for translating grammatically our experiments requires “ground truth” (GT) well-formed, edited text. The real-world, for- character files of the FL text and “reference trans- eign language (FL) document collections that lations” (RT) of the English text. In observing our our translators work with consist instead of team’s translator translate FL texts into RTs, we noisy and complex image files. We are cur- noticed the range of translation-support resources rently conducting experiments that involve and methods that he applied to the tasks. Given his building and evaluating the effectiveness of different multi-component workflows for the experience with semi-automated and fully auto- automated processing and translation of these mated MT metrics and his prior involvement in our FL images into English text. To support the design of MT user-support and evaluation tools, project’s ongoing needs for translations, we we decided to leverage his knowledge and involve are developing a software framework de- him directly in the creation of a toolkit that would signed with and for our translator that (i) assist him as well as other translators on our pro- streamlines users’ access to and capability to ject. While translators may rely on TM and other add-in and modify existing online tools and tools to help with productivity and consistency, data resources (ex. MT, TM, dictionaries, they lack a user-centered software framework for morphological analyzers, LM), (ii) builds per- assembling different categories of translation tools sistent data objects for later re-use, and (iii) provides users with a configuration screen and customizing their tools and translation screen. page to select the tools and data resources for In this paper, we begin with an overview of our their sessions, to set their tools’ options and software engineering approach and principles for display options for their translation screen designing a translator’s toolkit framework and then page. This paper introduces our extreme pro- present the categories of tools in use by our trans- gramming approach to the software engineer- lator for inclusion in the framework. We then re- ing of this new, hands-on translator’s view an example use of existing resources in a framework called TREAT (Toolkit and Re- translation task and conclude the paper with an source Environment to Assist Translation), introduction to the TREAT framework that we where the translator---as subject matter expert have implemented and continue to refine. and experienced software user---participates fully in the software design, evaluation, and iterative modification processes. 2 Design Approach The design approach for creating TREAT com- bines two separate, but complementary software engineering paradigms: extreme programming and user-centered design. This combination allows us teams developing software in the face of vague or to leverage our translator’s experience on different rapidly changing conditions”. (Beck, 1999) The translation tasks and in using different translation XP software engineering paradigm has been de- software. As depicted in Figure 1, we distinguish scribed by six principles: 1) embrace change, 2) three stakeholder roles in creating such a frame- keep the design simple, 3) minimize investment work: lead builder (developer), lead designer, and upfront, 4) support incremental change, 5) create user. In small-scale software development projects, functional releases of the software and 6) continu- the builder and the designer may be the same per- ally re-design based on user feedback. son and typically the builder/designer does not Extreme programming shortens application de- have the subject matter expertise of the intended velopment time by compressing the life-cycle and users. As a result, their understanding of the users’ sharing design, coding, and testing between two or tasks and the users’ software needs is several steps more software developers. Our work adheres to removed from the user, even when an extensive these principles both (i) with the choice of an in- requirements engineering process is undertaken to terpreted programming language (as opposed to document the user’s needs. compiled), such as JavaScript, Java, or Perl, to cre- By choosing an expert translator as lead de- ate rapidly revisable, functional software proto- signer of the toolkit framework, we quickly hear types with web pages as the user interface and (ii) back from the “user” when the implementation of with user feedback incrementally guiding which design desiderata by a developer does not support features to add, modify, or delete as the GUI for the intended use case. That is, the expert conducts framework software evolves. both the capability evaluation (validating that the User-centered design seeks to involve the end- software “does” its share, for example, providing user in all aspects of the software life-cycle, from correct data and analyses to support specific tasks) initial concept through operational fielding, includ- and the usability evaluation (determining that the ing the maintenance of a system through user feed- user can make use of the software features to effec- back and change requests. Variations of user- tively complete specific tasks). centered design, such as participatory design and contextual design, are all methods for building software that maps to user expectations and needs. With the lead design/expert translator in the role of the end user, we identified several principles to follow in the development of this framework: Keep it simple/short: keep all user-tool in- teractions as short and intuitive as possible Keep it simple/uncluttered: keep all user- views (GUIs) as open and uncovered as possible, ex. make sure tools and resources are non-intrusive: the tools should not get in the way of the translator Put the user in charge: enable user to hide or show tools, making them accessible if Figure 1 User-centered Design requested but suppressed if necessary. 2.1 Extreme Programming & User-Centered Support the translator-in-training: provide Design users with easy access to guide and hints. Provide extensibility for incorporating new Extreme Programming (XP) is the concept of cre- tools: enable user to easily add new tools ating software very quickly through the use of and expand framework’s capabilities rapid prototyping and incremental development. In Support customization by incorporating Extreme Programming Explained, XP is defined as new resources: enable experienced users to “a light-weight methodology for small- to medium- create their own data repositories and to 3.1 MT Engines and Multi-MT Tools import data of others MT engines are now available for fully automated 2.2 Progression of User Abilities and customized translation of digital texts. They vary widely in underlying construction methods Users do not remain at the same level of perform- (dictionary-based, symbolic rule-based, example- ance as they interact and gain familiarity with a based, phrase-based, syntax-based) and produce software application over time. By definition, an output translations that differ significantly in accu- experienced user is someone who has performed a racy and fluency. Although MT engines will fail in task enough times to have internalized the underly- domains that they were not built or trained to trans- ing rules or heuristics for effectiveness. We distin- late when compared against the translation quality guish two dimensions of “experience” among our of human translators, there are tasks (such as intended framework users: experience at transla- document triage and filtering) that do not require tion tasks and experience using software tools. full publication-quality, high-fluency in the target There are translators who are task experts, but language. Studies now document how MT engines have little or no software tool support. There are can be used effectively to assist translators on these translators-in-training who are novices at the task, tasks. (Bonet, 2009) but have been exposed to numerous tools. Our translator is an expert along both dimensions. There are also those translators that would fall in the middle in terms of using software and ability to translate foreign language texts. For example, non- native speakers of English who were educated in a foreign language may be expert translators into their language, but not into English. The framework currently supports four levels of users: beginning, learning, intermediate, and ad- vanced. Beginning user: Translates using default tools and tool settings “As-is”. Learning user: Makes choices and selects appropriate tool settings for using avail- able tools. Figure 2 Web client access to multiple MT engines Intermediate user: Customizes framework by adding their own data resources or set- Our research needs in building and evaluating mul- tings to unique tool configurations. tiple MT engines provided the catalyst for the de- velopment of MTriage, a front-end tool that Advanced user: Adds new tools to the communicates with several MT engines. (Hobbs et. framework to solve more complicated al., 2008) The web client for MTriage hides the tasks. details of configuration settings and transactions 3 Categories of Tools in the Framework with individual MT engines from the users, ena- bling them to focus on their translation task by There are many categories of tools at different lev- eliminating their need to learn the details of a sepa- els of maturity and capability that may provide rate GUI for each engine.