DOCSLIB.ORG

Large-Scale Discriminative N-Gram Language Models for Statistical Machine Translation

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Large-Scale Discriminative N-Gram Language Models for Statistical Machine Translation Large-scale Discriminative n-gram Language Models for Statistical Machine Translation Zhifei Li and Sanjeev Khudanpur Department of Computer Science and Center for Language and Speech Processing Johns Hopkins University, Baltimore, MD 21218, USA [email protected] and [email protected] Abstract is derived from a large corpus of text in the tar- get language via maximum likelihood estimation We extend discriminative n-gram language modeling techniques originally proposed for (MLE), in conjunction with some smoothing (Chen automatic speech recognition to a statistical and Goodman, 1998). In particular, the so called n- machine translation task. In this context, we gram model is particularly effective and has become propose a novel data selection method that the dominant LM in most systems. Several attempts leads to good models using a fraction of the have been made, particularly in speech recognition, training data. We carry out systematic ex- to improve LMs by appealing to more powerful es- periments on several benchmark tests for Chi- timation techniques, e.g., decision trees (Bahl et al., nese to English translation using a hierarchical phrase-based machine translation system, and 1989), maximum entropy (Rosenfeld, 1996), neural show that a discriminative language model networks (Bengio et al., 2001), and random forests significantly improves upon a state-of-the-art (Xu and Jelinek, 2004). Attempts have also been baseline. The experiments also highlight the made to extend beyond n-gram dependencies by ex- benefits of our data selection method. ploiting (hidden) syntax structure (Chelba and Je- linek, 2000) and semantic or topical dependencies 1 Introduction (Khudanpur and Wu, 2000). We limit ourselves to n-gram models in this paper, though the ideas pre- Recent research in statistical machine translation sented extend easily to the latter kinds of models as (SMT) has made remarkable progress by evolv- well. ing from word-based translation (Brown et al., 1993), through flat phrase-based translation (Koehn A regular LM obtained through the MLE tech- et al., 2003) and hierarchical phrase-based transla- nique is task-independent and tries to distinguish tion (Chiang, 2005; Chiang, 2007), to syntax-based between likely and unlikely word sequences with- translation (Galley et al., 2006). These systems usu- out considering the actual confusions that may ex- ally contain three major components: a translation ist in a particular task. This is sub-optimal, as model, a word-reordering model, and a language different tasks have different types of confusion. model. In this paper, we mainly focus on improv- For example, while the main confusion in a speech ing the language model (LM). recognition task is between similar sounding words A language model constitutes a crucial compo- (e.g., “red” versus “read”), the confusion in a ma- nent in many other tasks such as automatic speech chine translation task is mainly due to multiple word recognition, handwriting recognition, optical char- senses or to word order. An LM derived using an acter recognition, etc. It assigns a priori probabil- explicitly discriminative criterion has the potential ities to word sequences. In general, we expect a to resolve task-specific confusion: its parameters low probability for an ungrammatical or implausi- are tuned/adjusted by observing actual confusions in ble word sequence. Normally, a language model task-dependent outputs. Such discriminative n-gram language modeling has been investigated by Stolcke In SMT, x is a sentence in the source language, and Weintraub (1998), Chen et al. (2000), Kuo et al. GEN(x) enumerates possible translations of x into (2002), and Roark et al. (2007) on various speech the target language, and y is the desired translation: recognition tasks. It is scientifically interesting to either a reference translation produced by a bilingual see whether such techniques lead to improvement in human or the alternative in GEN(x) that is most an SMT task, given the substantial task differences. similar to such a reference translation. In the lat- We do so in this paper. ter case, y is often called the oracle-best (or simply We investigate application of the discriminative oracle) translation of x. n-gram language modeling framework of Roark et The components GEN(¢), ©, and ® define a map- al. (2007) to a large-scale SMT task. Our discrim- ping from an input x to an output y¤ through inative language model is trained using an aver- ¤ aged perceptron algorithm (Collins, 2002). In our y = arg max ©(x; y) ¢ ®; (1) y 2 GEN(x) task, there are millions of training examples avail- P able, and many of them may not be beneficial due where ©(x; y)¢® = j ®j©j(x; y), with j indexing to various reasons including noisy reference trans- the feature dimensions, is the inner product. lations resulting from automatic sentence-alignment Since y is a word-sequence, (1) is called global of a document-aligned bilingual text corpus. More- linear model to emphasize that the maximization is over, our discriminative model contains millions of jointly over the entire sentence y, not locally over features, making standard perceptron training pro- each word/phrase in y (as done in (Zens and Ney, hibitively expensive. To address these two issues, 2006; Chan et al., 2007; Carpuat and Wu, 2007)). we propose a novel data selection method that strives The learning task is to obtain the “optimal” pa- to obtain a comparable/better model using only a rameter vector ® from training examples, while the fraction of the training data. We carry out system- decoding task is to search, given an x, for the max- atic experiments on a state-of-the-art SMT system imizer y¤ of (1). These tasks are discussed next in (Chiang, 2007) for the Chinese to English transla- Section 2.2 and Section 2.3, respectively. tion task. Our results show that a discriminative LM is able to improve over a very strong baseline SMT 2.2 Parameter Estimation Methods system. The results also demonstrate the benefits of Given a set of training examples, the choice of ® our data selection method. may be guided, for instance, by an explicit criterion such as maximizing, among distributions in an expo- 2 Discriminative Language Modeling nential family parameterized by ©, the conditional i i We begin with the description of a general frame- log-likelihood of y given x . Algorithms that de- work for discriminative language modeling, recapit- termine ® in this manner typically operate in batch ulating for the sake of completeness the detailed de- mode—they require processing all the training data scription of these ideas in (Collins, 2002; Roark et (often repeatedly) before arriving at an answer—and al., 2007). require regularization techniques to prevent over- fitting, but are amenable to parallelized computing 2.1 Global Linear Models and often exhibit good empirical performance. A linear discriminant aims to learn a mapping from On the other hand, sequential algorithms such as an input x 2 X to an output y 2 Y , given the linear perceptron (Collins, 2002) operate in on- line mode—processing the training data sequentially 1. training examples (xi; yi); i = 1 ¢ ¢ ¢ N, to incrementally adjust the parameters—and are not 2. a representation © : X £ Y ! Rd mapping amenable to parallelization, but exhibit faster con- each possible (x; y) to a feature vector, vergence to parameter values that yield compara- ble empirical performance, particularly when large 3. a function GEN(x) ⊆ Y that enumerates puta- amounts of training data are available. In this paper, tive labels for each x 2 X, and we use the perceptron algorithm due to its simplicity 4. a vector ® 2 Rd of free parameters. and suitability to large data settings. Perceptron(x; GEN(x); y) Rerank-Nbest(GEN(x)) 1 ® à ~0 ¤ initialize as zero vector 1 y¤ à GEN(x)[1] ¤ baseline 1-best y 2 for t à 1 to T 2 for in GEN(x) P 3 for i à 1 to N 3 S(x; y) à ¯©0(x; y) + j2[1;F ] ®j©j(x; y) 4 z i à arg max ©(xi; z)¢ ® 4 if S(x; y) > S(x; y¤) i z2GEN(x ) 5 y¤ à y 5 if (zi 6= yi) 6 return y¤ 6 ® à ® + ©(xi; yi) ¡ ©(xi; zi) 7 return ® Figure 2: Discriminative LM reranking of the N-best list GEN(x) of a source sentence x. Figure 1: The Basic Perceptron Algorithm 2.3.1 Features used in the Discriminative LM 2.2.1 Averaged Perceptron Algorithm Each component ©j(x; y) of the feature vector Figure 1 depicts the perceptron algorithm (Roark can be any function of the input x and the output y. et al., 2007). Given a set of training examples, the al- We define the feature vector specific to our language gorithm sequentially iterates over the examples, and modeling task as follows. adjust the parameter vector ®. After iterating over Baseline Feature: We first define a baseline fea- the training data a few times, an averaged model, ture © (x; y), to be the score assigned to y by defined as 0 the baseline SMT system. This score itself is often a linear combination of several models, XT XN 1 1 i with the relative weights among these models ®avg = ® (2) T N t obtained via some minimum error rate training t=1 i=1 procedure (Och, 2003). i is computed and is used for testing, where ®t repre- Discriminative n-gram Features: The count of sents the parameter vector after seeing the i-th ex- each n-gram in y constitutes a feature. E.g., ample in the t-th iteration, N represents the size of the first n-gram feature may be, the training set, and T is the number of iterations the perceptron algorithm runs.
Load more

Recommended publications
  • A Challenge Set for Advancing Language Modeling
    A Challenge Set for Advancing Language Modeling Geoffrey Zweig and Chris J.C. Burges Microsoft Research Redmond, WA 98052 Abstract counts (Kneser and Ney, 1995; Chen and Good- man, 1999), multi-layer perceptrons (Schwenk and In this paper, we describe a new, publicly Gauvain, 2002; Schwenk, 2007) and maximum- available corpus intended to stimulate re- entropy models (Rosenfeld, 1997; Chen, 2009a; search into language modeling techniques Chen, 2009b). Trained on large amounts of data, which are sensitive to overall sentence coher- these methods have proven very effective in both ence. The task uses the Scholastic Aptitude Test’s sentence completion format. The test speech recognition and machine translation applica- set consists of 1040 sentences, each of which tions. is missing a content word. The goal is to select Concurrent with the refinement of N-gram model- the correct replacement from amongst five al- ing techniques, there has been an important stream ternates. In general, all of the options are syn- of research focused on the incorporation of syntac- tactically valid, and reasonable with respect to tic and semantic information (Chelba and Jelinek, local N-gram statistics. The set was gener- ated by using an N-gram language model to 1998; Chelba and Jelinek, 2000; Rosenfeld et al., generate a long list of likely words, given the 2001; Yamada and Knight, 2001; Khudanpur and immediate context. These options were then Wu, 2000; Wu and Khudanpur, 1999). Since in- hand-groomed, to identify four decoys which tuitively, language is about expressing meaning in are globally incoherent, yet syntactically cor- a highly structured syntactic form, it has come as rect.
    [Show full text]
  • ACL Lifetime Achievement Award
    ACL Lifetime Achievement Award The Dawn of Statistical ASR and MT Frederick Jelinek∗ Johns Hopkins University I am very grateful for the award you have bestowed on me. To understand your generosity I have to assume that you are honoring the leadership of three innovative groups that I headed in the last 47 years: at Cornell, IBM, and now at Johns Hopkins. You know my co-workers in the last two teams. The Cornell group was in Information Theory and included Toby Berger, Terrence Fine, and Neil J. A. Sloane (earlier my Ph.D. student), all of whom earned their own laurels. I was told that I should give an acceptance speech and was furnished with example texts by previous recipients. They wrote about the development and impact of their ideas. So I will tell you about my beginnings and motivations and then focus on the contributions of my IBM team. In this way the text will have some historical value and may clear up certain widely held misconceptions. 1. Beginnings Information Theory seemed to be one of the most prestigious disciplines during my years as a student at MIT (1954–1962). The faculty included the founders of the field— Shannon, Fano, Elias, and others. Some of my contemporaries were Viterbi, Jacobs, Kleinrock (founders of Qualcom), Gallagher, Kailath, and Massey. Not daring to ap- proach Shannon himself, I asked Professor Fano to be my thesis adviser. I was making slow progress when in 1961, after three years of trying, I succeeded in extricating my future wife Milena from communist Czechoslovakia (how this was accomplished is another story) and married her.
    [Show full text]
  • Improving Statistical Language Model Performance with Automatically Generated Word Hierarchies
    Improving Statistical Language Model Performance with Automatically Generated Word Hierarchies John G. McMahon* Francis J. Smith The Queen's University of Belfast An automatic word-classification system has been designed that uses word unigram and bigram frequency statistics to implement a binary top-down form of word clustering and employs an average class mutual information metric. Words are represented as structural tags--n-bit num- bers the most significant bit-patterns of which incorporate class information. The classification system has revealed some of the lexical structure of English, as well as some phonemic and se- mantic structure. The system has been compared---directly and indirectly--with other recent word-classification systems. We see our classification as a means towards the end of construct- ing multilevel class-based interpolated language models. We have built some of these models and carried out experiments that show a 7% drop in test set perplexity compared to a standard interpolated trigram language model. 1. Introduction Many applications that process natural language can be enhanced by incorporating information about the probabilities of word strings; that is, by using statistical language model information (Church et al. 1991; Church and Mercer 1993; Gale, Church, and Yarowsky 1992; Liddy and Paik 1992). For example, speech recognition systems often require some model of the prior likelihood of a given utterance (Jelinek 1976). For convenience, the quality of these components can be measured by test set perplexity, PP (Bahl, Jelinek, and Mercer 1983; Bahl et al. 1989; Jelinek, Mercer, and Roukos 1990), in spite of some limitations (Ueberla 1994): PP = P(wlN)- ~, where there are N words in the word stream (w~/and ib is some estimate of the probability of that word stream.
    [Show full text]
  • NOTE Our Lucky Moments with Frederick Jelinek
    The Prague Bulletin of Mathematical Linguistics NUMBER 88 DECEMBER 2007 91–92 NOTE Our Lucky Moments with Frederick Jelinek Barbora Vidová Hladká is contribution is going to be a congratulation to Frederick Jelinek’s birthday jubilee. Be- fore I reach the very congratulation I would like to remind a lucky moment that had a strong influence on the life of a certain Institute of Charles University in Prague aer 1989. And it is by no chance that the honored person witnessed the above mentioned moment and its con- sequences. From my personal point of view, I have become one of the ”victims” of this lucky moment so I really appreciate the opportunity to wish well to Fred via the Prague Bulletin circulating the institutions over the world. e crucial events in November 1989 in Czech Republic brought freedom to a lot of people. Freedom to scientists in the group of computational linguistics at the Faculty of Mathematics and Physics, Charles University changed (among other things) their subdepartment into an independent department of the faculty in 1990, namely the Institute of Formal and Applied Linguistics (ÚFAL) headed by Eva Hajičová. Freedom to Fred Jelinek made it possible for him (among other things) to give a two term course on spoken and written language analysis at the Czech Technical University in Prague in 1991-1992. At that time, Fred was a senior manager of the IBM T.J. Watson Research Center, Yorktown Heights, NY and he was heading a group carrying out research on continuous speech recognition, machine translation and text parsing and understanding.
    [Show full text]
  • Fred Jelinek
    Obituary Fred Jelinek Mark Liberman University of Pennsylvania Frederick Jelinek died, peacefully and unexpectedly, on 14 September 2010. Over a distinguished career of nearly fifty years, Fred made important contributions in areas ranging from coding theory and speech recognition to parsing and machine translation. But more important than these specific contributions was his role in persuading the fields of speech and language engineering to adopt statistical methods and the “noisy channel model,” returning to the path opened up by Claude Shannon in 1948. And more important still was the role that he played in defining, exemplifying, and promoting what has become the standard research paradigm in an even wider range of fields: the competitive evaluation of alternative algorithms based on precise quantitative criteria defined in advance, relative to a shared body of training and testing material. After receiving his Ph.D. from MIT in 1962, Fred taught at Cornell from 1962 to 1972, worked at IBM from 1972 to 1993, and taught at Johns Hopkins from 1993 to 2010. Fred’s many technical accomplishments during this long and productive career can be seen as episodes in two epic narratives, which, like the Iliad and the Odyssey, are related but have separate themes and story lines. The theme of the first epic is the return of Information Theory to center stage in speech and language processing; and the theme of the second epic is the development of a new relationship between science and engineering in speech recognition, computational linguistics, and artificial intelligence (AI) more generally. Fred gave us a vivid first-person narrative of the first of these epic adventures in his ACL Lifetime Achievement Award speech (Jelinek 2009).
    [Show full text]
  • Corpus Methods in a Digitized World
    Corpus Methods in a Digitized World Kenneth Church Baidu, USA Sampling (Teaser) • Sampling Desiderata: Corpus ≈ Sample – Corpus should be representative of pop of interest – Miserable Failure: Academic Books ≠ Chap’s Interests • Quality (representative) vs. Quantity (more): – More is more??? • But in a digitized world (where we have it all) – Do we still need to worry about sampling? • If a corpus is comprehensive, – does that imply that it is balanced? Dec 17, 2018 2 Population Bound • Corpus data is limited by population – There are only 7B people – And they have only so much time to communicate – And only so much to say • It is becoming technically possible to capture “much” of this (a non-trivial fraction) – Google Ngrams: 4% of all books – Nanny Cams: all speech that babies are exposed to • If our corpora are large (comprehensive) – Does that make sampling (balance) moot? Dec 17, 2018 3 Blooper Risk • Search • Auto-complete • Spelling correction • Ads • Chatbots • Memes Dec 17, 2018 4 However, at one point Tay tweeted about taking drugs, in front of the police, no less. https://www.theguardian.com/technology /2016/mar/30/microsoft-racist-sexist- chatbot-twitter-drugs Dec 17, 2018 5 Dec 17, 2018 6 Preventing Bloopers • (Taboo) MWEs: – common verb + function word • go, make, do, have, give, call • it, up, in, on, with, out, down, around, over • Amusing failure mode for Yarowsky (1992) Grolier's Roget’s (Chap) Dec 17, 2018 7 Rip-Roaring, Vast, Vetted & Varied Zany Zappy Dec 17, 2018 Grolier's Chap 8 “Field Work” / Guilty Pleasure Dec 17, 2018 9 Chap & Technology (& me) http://www.encyclopedia.com/arts/educational-magazines/chapman-robert-lundquist-1920-2002 • Chapman edited the fourth edition in 1977, – but it was his fifth edition, published in 1992, – that expanded the compendium with more than 50,000 new words – including colloquialisms such as • "AIDS," "yuppie," "hacker," • and "crack" (as in cocaine) – that were unknown in Roget's time.
    [Show full text]
  • 26Automatic Speech Recognition and Text-To-Speech
    Speech and Language Processing. Daniel Jurafsky & James H. Martin. Copyright © 2021. All rights reserved. Draft of September 21, 2021. CHAPTER Automatic Speech Recognition 26 and Text-to-Speech I KNOW not whether I see your meaning: if I do, it lies Upon the wordy wavelets of your voice, Dim as an evening shadow in a brook, Thomas Lovell Beddoes, 1851 Understanding spoken language, or at least transcribing the words into writing, is one of the earliest goals of computer language processing. In fact, speech processing predates the computer by many decades! The first machine that recognized speech was a toy from the 1920s. “Radio Rex”, shown to the right, was a celluloid dog that moved (by means of a spring) when the spring was released by 500 Hz acous- tic energy. Since 500 Hz is roughly the first formant of the vowel [eh] in “Rex”, Rex seemed to come when he was called (David, Jr. and Selfridge, 1962). In modern times, we expect more of our automatic systems. The task of auto- ASR matic speech recognition (ASR) is to map any waveform like this: to the appropriate string of words: It's time for lunch! Automatic transcription of speech by any speaker in any environment is still far from solved, but ASR technology has matured to the point where it is now viable for many practical tasks. Speech is a natural interface for communicating with smart home ap- pliances, personal assistants, or cellphones, where keyboards are less convenient, in telephony applications like call-routing (“Accounting, please”) or in sophisticated dialogue applications (“I’d like to change the return date of my flight”).
    [Show full text]
  • Lecture 5: N-Gram Language Models
    Speech Recognition Lecture 5: N-gram Language Models Mehryar Mohri Courant Institute and Google Research [email protected] Language Models Definition: probability distribution Pr[ w ] over sequences of words w = w1 ...wk. • Critical component of a speech recognition system. Problems: • Learning: use large text corpus (e.g., several million words) to estimate Pr[ w ] . Models in this course: n-gram models, maximum entropy models. • Efficiency: computational representation and use. Mehryar Mohri - Speech Recognition page 2 Courant Institute, NYU This Lecture n-gram models definition and problems Good-Turing estimate Smoothing techniques Evaluation Representation of n-gram models Shrinking LMs based on probabilistic automata Mehryar Mohri - Speech Recognition page 3 Courant Institute, NYU N-Gram Models Definition: an n-gram model is a probability distribution based on the nth order Markov assumption ∀i, Pr[wi | w1 ...wi−1]=Pr[wi | hi], |hi|≤n − 1. • Most widely used language models. Consequence: by the chain rule, k k Pr[w]=! Pr[wi | w1 ...wi−1]=! Pr[wi | hi]. i=1 i=1 Mehryar Mohri - Speech Recognition page 4 Courant Institute, NYU Maximum Likelihood Likelihood: probability of observing sample under distribution p ∈ P , which, given the independence assumption is m Pr[x1,...,xm]=! p(xi). i=1 Principle: select distribution maximizing sample probability m p! = argmax ! p(xi), ∈P p i=1 m or p! = argmax ! log p(xi). ∈P p i=1 Mehryar Mohri - Speech Recognition page 5 Courant Institute, NYU Example: Bernoulli Trials Problem: find most likely Bernoulli distribution, given sequence of coin flips H, T, T, H, T, H, T, H, H, H, T, T, .
    [Show full text]
  • History and Survey of AI August 31, 2012
    CS671 Artificial General Intelligence History and Survey of AI August 31, 2012 Where did it go wrong and where did it go right. 1 Observation: AI has made steady progress since its inception about 60 years ago. This is contrary to some accounts in the popular press. 2 The \Gullibility Wave" Phenomenon • First reaction to AI: Skepticism • Second (perhaps having seen a robot demo or learned to pro- gram a computer): Wow! Computers can do a lot. Gullibility Doing intelligent task X isn't so hard. When I think of doing it myself, it seems straightforward! Enthusiasm • Third reaction: Computers are impossibly literal-minded. They can never break out of this mold. Disenchantment • Fourth reaction: The brain doesn't do computation anyway. It does holograms, or dynamical systems, or quantum gravity, or neural nets, or soulstuff. Denial • Fifth reaction: There's really no alternative on the horizon to computation (which is what neural nets actually do). Let's get to work! Maturity 3 The Beginning of Computer Science Was the Beginning of AI Alan Turing (1912{1954) invented both. Perhaps it was because he was weird, but even before the end of World War II he discussed machine intelligence with col- leagues at the British Code and Cipher School (\Bletch- ley Park"). 4 1940s: A Gleam In Turing's Eye In 1950 Alan Turing published \Computing Machinery and Intelligence." (Mind 49) He predicted that a machine would pass the Turing Test by 2000 (i.e., would fool a (naive?) judge 70% of the time into thinking it was human after a short interview).
    [Show full text]
  • A Maximum Likelihood Approach to Continuous Speech Recognition
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-5, NO. 2, MARCH 1983 179 A Maximum Likelihood Approach to Continuous Speech Recognition LALIT R. BAHL, MEMBER, IEEE, FREDERICK JELINEK, FELLOW, IEEE, AND ROBERT L. MERCER Abstract-Speech recognition is formulated as a problem of maximum likelihood decoding. This formulation requires statistical models of the TE T ACOUSTIC LINGUISTIC speech production process. In this paper, we describe a number of sta- GENERATOR ----aSPEAKER PROCESSOR __.DECODER tistical models for use in speech recognition. We give special attention to determining the parameters for such models from sparse data. We SPEECH RECOGNIZER also describe two decoding methods, one appropriate for constrained Fig. 1. A continuous speech recognition system. artificial languages and one appropriate for more realistic decoding tasks. To illustrate the usefulness of the methods described, we review a number of decoding results that have been obtained with them. TEXT SPEAKER ACOUSTIC LINGUISTIC w Index Terms-Markov models, maximum likelihood, parameter esti- GENERATOR PROCESSOR DECODER mation, speech recognition, statistical models. ACOUSTIC CHANNEL Fig. 2. The communication theory view of speech recognition. I. INTRODUCTION to the task domain, THE AIM of research in automatic speech recognition is In addition the constraint imposed by the experimental environment is often restricted in several the development of a device that transcribes natural speech automatically. Three areas of speech recognition research can other ways. For example, at IBM speech is recorded with a be distinguished: 1) isolated word recognition where words are headset microphone; the system is tuned to a single talker; the separated by distinct pauses; 2) continuous speech recognition talker is prompted by a script, false starts are eliminated, etc.; many time for each where sentences are produced continuously in a natural man- recognition often requires seconds of CPU ner; and 3) speech understanding where the aim is not tran- second of speech.
    [Show full text]
  • Some of My Best Friends Are Linguists
    Some of my Best Friends are Linguists (LREC 2004) Frederick Jelinek Johns Hopkins University THANKS TO: E. Brill, L. Burzio, W. Byrne, C. Cieri, J. Eisner, R. Frank, L. Guthrie, S. Khudanpur, G. Leech, M. Liberman, M. Marcus, M. Palmer, P. Smolensky, and D. Yarowsky May 28, 2004 Johns Hopkins The Quote “Whenever I fire a linguist our system performance improves” From my talk entitled: Applying Information Theoretic Methods: Evaluation of Grammar Quality Workshop on Evaluation of NLP Systems, Wayne PA, December 1988 May 28, 2004 Johns Hopkins Hope Springs Eternal • My colleagues and I always hoped that linguistics will eventually allow us to strike gold – HMM tagging (Bahl & Mercer 1976) • The quote accentuated a certain situation that existed in ASR in the seventies and in NLP in the eighties • The following is an illustration May 28, 2004 Johns Hopkins The Raleigh Finite State Language May 28, 2004 Johns Hopkins Zoom on Raleigh Language May 28, 2004 Johns Hopkins When Linguists Left the Group Task: New Raleigh Language • Acoustic model 1: – phonetic baseforms: three ßà ?rí – Model statistics estimated by experts (35% accuracy) • Acoustic model 2: – phonetic baseforms: three ßà ?rí – Model statistics estimated automatically from data (75% accuracy) • Acoustic model 3: – orthographic baseforms: three ßà THREE – Model statistics estimated automatically from data (43% accuracy) May 28, 2004 Johns Hopkins Judgment of J.R. Pierce “ Whither Speech Recognition?” JASA 1969 …ASR is attractive to money. The attraction is perhaps similar to the attraction of schemes for turning water into gasoline, extracting gold from the sea, or going to the moon.
    [Show full text]
  • Ieee-Level Awards
    IEEE-LEVEL AWARDS The IEEE currently bestows a Medal of Honor, fifteen Medals, thirty-three Technical Field Awards, two IEEE Service Awards, two Corporate Recognitions, two Prize Paper Awards, Honorary Memberships, one Scholarship, one Fellowship, and a Staff Award. The awards and their past recipients are listed below. Citations are available via the “Award Recipients with Citations” links within the information below. Nomination information for each award can be found by visiting the IEEE Awards Web page www.ieee.org/awards or by clicking on the award names below. Links are also available via the Recipient/Citation documents. MEDAL OF HONOR Ernst A. Guillemin 1961 Edward V. Appleton 1962 Award Recipients with Citations (PDF, 26 KB) John H. Hammond, Jr. 1963 George C. Southworth 1963 The IEEE Medal of Honor is the highest IEEE Harold A. Wheeler 1964 award. The Medal was established in 1917 and Claude E. Shannon 1966 Charles H. Townes 1967 is awarded for an exceptional contribution or an Gordon K. Teal 1968 extraordinary career in the IEEE fields of Edward L. Ginzton 1969 interest. The IEEE Medal of Honor is the highest Dennis Gabor 1970 IEEE award. The candidate need not be a John Bardeen 1971 Jay W. Forrester 1972 member of the IEEE. The IEEE Medal of Honor Rudolf Kompfner 1973 is sponsored by the IEEE Foundation. Rudolf E. Kalman 1974 John R. Pierce 1975 E. H. Armstrong 1917 H. Earle Vaughan 1977 E. F. W. Alexanderson 1919 Robert N. Noyce 1978 Guglielmo Marconi 1920 Richard Bellman 1979 R. A. Fessenden 1921 William Shockley 1980 Lee deforest 1922 Sidney Darlington 1981 John Stone-Stone 1923 John Wilder Tukey 1982 M.
    [Show full text]