sign in sign up
<<
Home , Machine translation

Generation for Statistical Machine

Ann Clifton Anoop Sarkar Simon Fraser University Simon Fraser University Burnaby, BC, Canada Burnaby, BC, Canada [email protected] [email protected]

1 Introduction follow (Minkov et al., 2007; Toutanova et al., 2008), who use MEMMs to predict inflected forms State of the art Machine Translation (MT) systems from sets of inflected surface forms which have tend to perform poorly when translating into lan- been gathered to correspond to each stem. We use guages with rich morphological systems. When CRFs instead of MEMMs and we use a smaller set multiple morphemes inflect a single stem, of features, dropping the -based features. the total of surface forms can be very To isolate the generation task, we begin by large, and any given surface form may occur in- the reference and using the frequently in a text, leading to a significant data CRF to predict the morphological tag sequences sparsity problem. for the stems. For the full translation task, we ob- In this paper, we examine how morphological tain the surface forms corresponding to the tags information incorporated into MT models can be by using n-gram model disambiguation used to address this problem. Our first approach (over morphemes) in surface form prediction. In is adapted from previous work (Popovic´ and Ney, the full pipeline, we train a phrase-based MT sys- 2004; de Gispert and Marino,˜ 2008), which seg- tem on stemmed text and use our CRF model to ments the morphologically complex language into predict the inflection tag sequences on the MT out- stems and suffixes and then trains the MT system put stems, and from the inflection tags produce on the segmentations, treating each as an individ- the surface word using the . We ual word; the motivation is that productive stems then combined the two approaches using the CRF and suffixes in isolation may reduce sparsity, as model to predict suffixes for hanging stems in the well as increase the level of lexical symmetry be- segmentation-trained MT output. tween source and target in the translation model. The second approach we use was proposed in English to Finnish We chose Finnish as our tar- (Minkov et al., 2007; Toutanova et al., 2008), in get language because of the richness of its mor- which complex in the target are stemmed phological system and because MT systems per- before translation, and then the morphologically formed most poorly on producing Finnish (Koehn, inflected forms are generated for the target stem 2005). We used English-Finnish data from the Eu- output as a post-processing step by predicting a roparl data set and trained all four MT models on sequence of morphemes using a Maximum En- those sentences 40 words or less, 977335/943159 tropy Markov Model (MEMM). We implemented sentences for the unsupervised/supervised seg- this approach using Conditional Random Fields mented data respectively, and 986166 sentences (CRFs) instead of MEMMs. for the word and stem-only data. The test data was also from Europarl, the same set as in (Koehn, 2 Approach and Experiments 2005) with 2000 sentences. In our first set of experiments, we performed both Morpheme segmentation To derive the stem supervised and unsupervised segmentation on the and morphological information by supervised data, onto which we added word-internal morphol- means, we ran the Omorfi FST morphological an- ogy boundary markers for recovery of the surface alyzer (Omorfi, 2007) on the data, from which we forms after decoding; we then used this segmented extracted for each word its lemma, part-of-speech data to train a phrase-based MT system. (POS) tag, and morphologically decomposed in- For the morphology-generation approach, we flection form. When no analysis was generated, we retained the unstemmed word with null fea- Model BLEU score ture values. We used the FST analyzer in hopes Baseline-words 14.39 that it might be more effective at yielding produc- USV segs 14.94 tive morphological patterns in fusional type lan- SV segs 14.58 guages with a great degree of morphophonemic al- SV stems with tag-CRF 10.09 ternation that make straightforward segmentation SV segs with suffix-CRF 14.58 difficult. To derive the unsupervised segmenta- tion for greater coverage and generality, we used Table 1: Model scores. USV/SV refer to unsuper- Morfessor (Creutz and Lagus, 2005). Using these vised/supervised; segs refer to models trained on segmentations, we trained four phrase-based MT segmentation with morphology in decoding; stems models (in addition to the word-trained baseline refers to models trained on stems alone with mor- model): the supervised and unsupervised stem and phology generated in post-processing. suffix segmentations, and a model trained on the stems alone. CRF-based morpheme prediction We trained performed the baseline word-based model. the CRF model on lexical and morphological fea- For the evaluation of the CRF morphology pre- tures using CRFSGD (Bottou, 2007). For the lexi- diction model, we first compared the output of cal features, we considered unigrams and the CRF model on the stemmed reference trans- of up to two of the previous and next lemmas and lations to the original inflection sequences gener- POS-tags, as well as the unigrams and bigrams ated by the morphological analyzer before apply- of the previous two POS-tags for prediction bi- ing the language model to predict the correspond- grams. For the tag sequence morphological fea- ing surface forms. The best results were obtained tures, we represented each of the series of morpho- using features that were conjunctions of predic- logical tags making up the inflection prediction set tion bigrams and observation context. For the as a vector of feature values corresponding to each CRF trained on tag sequences, the accuracy was morphological category, using the categories of 83.85% and 79.18%. number, case, and person. To recover the fully in- We then looked at the performance on MT sys- flected word forms corresponding to the CRF out- tem output, to see if the performance gains of put lemma plus inflection tag sequence, we used translating stems would outweigh errors in sur- a word-based language model to predict which in- face form prediction. For the tag-sequence CRF, flected word should correspond to an ambiguous this was evaluated after using the language model lemma plus tag sequence. For the tag-sequence to recover the surface forms. When we applied CRF training set, we used 315,247 Finnish sen- the CRF tag/suffix prediction model to the hang- tences and the same 2000 sentence test set. ing stems in the segmentation-trained MT output, We tested the CRF morphology prediction the BLEU scores remained the same. model on a stemmed version of a Finnish text as well as on the (ostensibly noisier) output of the We found that the models that use the morphol- Moses MT system trained on stemmed input from ogy in translation was able to improve the BLEU the supervised analysis. Since the unsupervised scores over the baseline. In addition, that higher stems-only model performed more poorly than the accuracy in inflection-tag sequence prediction was baseline, we did not use this output to test the mor- obtained using overlapping features would seem to phology generation post-processing. confirm the utility of using a CRF model for cap- We then tested the CRF suffix prediction model turing dependencies between predictions. on the output of the MT system trained on seg- To improve upon the results for using the CRF mented stems and suffixes, applying it only to to boost the segmented MT output, we are cur- hanging stems translated without a following suf- rently running experiments that include augment- fix. ing the CRF model feature set with lexical and syntactic bilingual features of the aligned corpora, 3 Results and Ongoing Work and using discontinuous phrase-based and lattice- Table 1 shows that the MT models trained on both based decoding. supervised and unsupervised segmentations out- References Leon Bottou. Stochastic Gradient CRFs homepage. http://leon.bottou.org/projects/sgd. 2007 Mathias Creutz and Krista Lagus. Inducing the mor- phological lexicon of a from unan- notated text. Proceedings of the International and Interdisciplinary Conference on Adaptive Knowl- edge Representation and Reasoning, 2005.

Adria de Gispert and Jose´ Marino.˜ On the impact of morphology in English to Spanish statistical MT. Speech Communication, 2008. Phillip Koehn. Europarl: a parallel corpus for statisti- cal machine translation. Proceedings of MT Summit, 2005. Phillip Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. Moses: Open source toolkit for statistical machine translation Annual Meeting of ACL, demonstration session, 2007.

John Lafferty, Andrew McCallum, and Fernando Pereira. Conditional random fields: probabilistic models for segmenting and labeling data. Proceed- ings of the 18th International Conference on Ma- chine Learning, 2001.

Andrew McCallum, Dayne Freitag, and Fernando Pereira. Maximum entropy markov models for and retrieval. Proceedings of the 17th International Conference on , 2000. Einat Minkov, Kristina Toutanova, and Hisami Suzuki. Generating complex morphology for machine trans- lation. Association for Computational , 2007. Maja Popovic´ and Hermann Ney. Towards the use of word stems and suffixes for statistical machine trans- lation. 4th International Conference on Language Resources and Evaluation (LREC), 2004. Hisami Suzuki and Kristina Toutanova. Learning to predict case markers in Japanese. Association for Computational Linguistics, 2006. Kristina Toutanova and Hisami Suzuki. Generating case markers in machine translation. Association for Computational Linguistics, 2007.

Kristina Toutanova, Hisami Suzuki, and Achim Ruopp. Applying morphology generation models to ma- chine translation Proceedings of ACL-08: HLT, 2008. http://kitwiki.csc.fi/twiki/bin/view/KitWiki/OMorFiHome 2007.