DOCSLIB.ORG

Multimodal Semantic Understanding: Semantic Role Labeling with Vision and Language

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Multimodal Semantic Understanding: Semantic Role Labeling with Vision and Language Multimodal Semantic Understanding: Semantic Role Labeling with Vision and Language by Zijiao Yang B.E., Dalian University of Technology 2016 B.E., Ritsumeikan University, 2016 A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Master of Science Department of Computer Science 2020 Committee Members: James Martin Martha Palmer Chenhao Tan ii Yang, Zijiao (M.S., Computer Science) Multimodal Semantic Understanding: Semantic Role Labeling with Vision and Language Thesis directed by Prof. James Martin Semantic role labeling (SRL) is an important task in natural language understanding systems. Recently, researchers have moved from traditional syntactic feature based pipelines to end-to-end (merely token to tag, mostly) neural network based SRL system, and state-of-the-art performance has been reported from those systems. Nonetheless, the recent trend of inducing syntactic infor- mation back to neural models has led to a big success [Strubell et al., 2018]. On the other hand, language understanding should not be a single modal task, the most pervasive evidence is our- selves. Incorporating information from other modalities has drawn attention [Baltrusaitis et al., 2019]. This thesis introduces two different models trying to utilize image information to help SRL task. The two models use distinct ways to integrate between the vision and language modalities. Also, the models are trained on automatically generated data, and evaluated on ground truth data. The results and analysis are presented. Dedication To my mom, dad, my girlfriend Yuping, and all my friends. iv Acknowledgements I want to thank Professor. James Martin, for his help throughout my thesis process, as well as his insightful opinions on various other projects that could be related to this thesis. I would also like to thank my committee members. This thesis originates from a class project of Professor Martha Palmer's Computational Lexical Semantics class. And Professor Martha Palmer provided me valuable feedback on related topics and connects me with her student Abhidip Bhattacharyya. Also they helped with a lot problems regrading the datasets I used. I thank Professor Chenhao Tan for his valuable comments and feedback and some most important skills I learned for proceeding this thesis is within a class with him. Finally, I want to thank Abhidip Bhattacharyya for the valuable discussions we had and the tools he pointed me to. v Contents Chapter 1 Introduction 1 1.1 The Necessity of Knowledge Enrichment to NLP Systems . .1 1.2 Task Selection . .3 1.3 Goal Description and Question Proposition . .4 1.4 Thesis Outline . .4 2 Previous Work 5 2.1 Semantic Role Labeling . .5 2.2 Multimodal Learning . 10 3 Data Description 14 3.1 Datasets Description . 14 3.2 Training and Validation Data Generation . 15 3.3 Test Data Description . 18 4 Model Description 20 4.1 Text Model . 21 4.1.1 Task Formulation . 21 4.1.2 BiLSTM . 22 4.1.3 Highway Connections . 24 vi 4.1.4 Decoding . 26 4.2 Alignment Model . 26 4.2.1 Image Module . 26 4.2.2 Alignment Module . 28 4.2.3 Loss . 29 4.3 Attention Model . 29 4.3.1 Attention Module . 30 4.4 Training Setup . 30 5 Results and Analysis 33 5.1 Results . 33 5.2 Analysis . 34 5.2.1 Ablation . 34 5.2.2 Confusion Matrix . 35 5.2.3 Attention and Decoding . 36 6 Conclusion and Future Work 49 6.1 Conclusion . 49 6.2 Future Work . 50 Bibliography 52 vii Tables Table 2.1 Some of the common modifiers in current Propbank (not a complete list, only include the semantic role concerned in this thesis) [Bonial et al., 2012], each row is a modifier, its short definition, and one example. Attention on the emphasized texts. .8 5.1 This table gives the average precision, recall and F1 over five training instances with different random initialization for t-SRL, al-SRL, at-SRL. In side the brackets, the standard deviation across 5 training instances is reported . 34 5.2 This table gives the precision, recall, and f1 for each semantic role label, along with overall score for t-SRL model results (This is not a complete table) . 35 5.3 This table gives the precision, recall, and f1 for each semantic role label, along with overall score for al-SRL model results (This is not a complete table) . 36 5.4 This table gives the precision, recall, and f1 for each semantic role label, along with overall score for at-SRL model results (This is not a complete table) . 37 5.5 This table gives the result of ablation, involved models are the baseline model t-SRL, at-SRL, at-SRL without highway connection (no hi), and at-SRL without decoding (no decoding), the precision, recall and F1 for each model is reported. The number in brackets represent the F1 drop compared to at-SRL . 38 5.6 This table gives the precision, recall and F1 score with respect to individual semantic role labels as well as overall results for t-SRL. It is the computed based on the predicted results without decoding (it is not a complete table) . 39 viii 5.7 This table gives the precision, recall and F1 score with respect to individual semantic role labels as well as overall results for al-SRL. It is the computed based on the predicted results without decoding (it is not a complete table) . 39 5.8 This table gives the precision, recall and F1 score with respect to individual semantic role labels as well as overall results for at-SRL. It is the computed based on the predicted results without decoding (it is not a complete table) . 45 5.9 Performance drop without decoder on precision, recall, F-1, for t-SRL, al-SRL, and at-SRL . 45 ix Figures Figure 2.1 An example of span-based SRL . .6 3.1 An example from MSCOCO dataset. Each image is paired with five captions . 16 3.2 An example from Flickr30k dataset. The image data contains bounding boxes de- noted by color-coded squares. (The bounding boxes shown here is from original Flickr30k dataset, which is human annotated, but the bounding box features used in this thesis are generated follow [Tan and Bansal, 2019]) The entities in the captions are also color-coded. Same color indicates correspondences between bounding boxes and entities mentioned in captions. 17 4.1 3 layer BiLSTM with highway connections, pink arrow denotes the highway connec- tion; the red boxes are the gates that control the integration of highway knowledge and hidden states. 22 4.2 The alignment model al-SRL: it consists of three parts: image module(bottom left, blue box), BiLSTM (bottom right), and alignment module (where the alignment loss is computed). A sentence embedding is extracted from BiLSTM hidden states and resized image embedding to be fed into the alignment module for computing alignment loss. Two losses are integrated to form final loss. 27 x 4.3 The attention module: the initial layers of text part are identical to t-SRL4.1. Bound- ing box features are obtained by passing the concatenation of bounding box vector and coordinates to the image module, as described in 4.2.1. Then context vector is computed for each hidden state by attention module. Finally, the context vectors and hidden states are concatenated together to make final predication. 32 5.1 The confusion matrix for t-SRL, (note it is a simplified version of complete confusion matrix. Darker color denotes more mistakes, each celli;j represents should be label i, but predicted as label j . 40 5.2 The confusion matrix for al-SRL, (note it is a simplified version of complete confusion matrix. Darker color denotes more mistakes, each celli;j represents should be label i, but predicted as label j . 41 5.3 The confusion matrix for at-SRL. (note it is a simplified version of the complete confusion matrix.) Darker color denotes more mistakes, each celli;j represents should be label i, but predicted as label j . 42 5.4 This heatmap represents the confusion difference between t-SRL and at-SRL with- out decoding. It is computed by subtracting the confusion matrix of at-SRL (no decoding) from t-SRL (no decoding). Blue color represents the confusion decrease for at-SRL compared with t-SRL; for example, cellLOC;ARG2 represents at-SRL is less confused about LOC with ARG2 by 11. Furthermore, red color means the confusion increase for at-SRL compared with t-SRL. 43 5.5 This heatmap represents the confusion difference between t-SRL and at-SRL with decoding. It is computed by subtracting the confusion matrix of at-SRL from t-SRL. Blue color represents the confusion decrease for at-SRL compared with t-SRL; for example, cellLOC;ARG2 represents at-SRL is less confused about LOC with ARG2 by 14. Furthermore, red color means the confusion increase for at-SRL compared with t-SRL. 44 xi 5.6 The confusion matrix for t-SRL without decoding, (note it is a simplified version of complete confusion matrix. Darker color denotes more mistakes, each celli;j rep- resents should be label i, but predicted as label j . 46 5.7 The confusion matrix for al-SRL without decoding, (note it is a simplified ver- sion of complete confusion matrix. Darker color denotes more mistakes, each celli;j represents should be label i, but predicted as label j . 47 5.8 The confusion matrix for at-SRL without decoding, (note it is a simplified ver- sion of complete confusion matrix.
Load more

Recommended publications
  • Dan Jurafsky: Curriculum Vitae
    Dan Jurafsky Department of Linguistics & [email protected] Department of Computer Science, http://www.stanford.edu/~jurafsky Stanford University Stanford CA 94305-2150 Education B.A Linguistics, with honors, University of California at Berkeley, 1983 Ph.D. Computer Science, University of California at Berkeley, 1992 Postdoc, International Computer Science Institute, Berkeley, 1992-1995 Academic Employment Stanford University: Professor and Chair of Linguistics and Professor of Computer Science, 2014- Stanford University: Professor of Linguistics and (by courtesy) of Computer Science, 2010- Stanford University: Associate Professor of Linguistics and (by courtesy) of Computer Science, 2004-2010 University of Colorado, Associate Professor of Linguistics, Computer Science, Cognitive Science, 2001-2003 University of Colorado, Assistant Professor of Linguistics, Computer Science, and Cognitive Science, 1996-2001 Honors, Awards, & Fellowships Fillmore Professor, Linguistics Society of America 2015 James Beard Foundation Book Award Finalist for The Language of Food, 2015 Fellow, Center for Advanced Study in the Behavioral Sciences, 2012-2013 MacArthur Fellowship, 2003 NSF CAREER Award, 1998 Paper Awards: 2015 Roger V. Gould Prize (McFarland et al 2013); Best Paper WWW 2013 (Danescu-Niculescu- Mizil et al 2013); Best Paper EMNLP 2013 (Spitkovsky, Alshawi, Jurafsky 2013); Best Paper ACL 2006 (Snow, Jurafsky, and Ng 2006); Distinguished paper IJCAI 2001 (Gildea & Jurafsky 2001); Cogsci 1998 Marr Prize Hon- orable Mention (Narayanan and Jurafsky 1998) Awards for student advisees Robert M. Golden Medal for outstanding undergraduate thesis: Justine Kao, Stanford, 2011 Robert M. Golden Medal for outstanding undergraduate thesis: Richard Futrell, Stanford, 2010 Firestone Medal for outstanding undergraduate thesis: David Hall, Stanford, 2008 Outstanding Student Paper Award: Gregory, Raymond, Bell, Jurafsky (1999), Society for Text and Discourse Research Books and Edited Volumes 1.
    [Show full text]
  • GAZETTE Dan Jurafsky
    the newsletter of the International Computer Science Institute The ICSI GAZETTE volume two | issue two | march 2004 featured alum: Dan Jurafsky by John Tortorice Dan Jurafsky, outstanding ICSI alum, has had many triumphs - but not everything works out in science, not even for him. His first setback in technology came before he, or many others, had even set eyes on a computer. “When I was about 12 years old (in the 1970s), I read a newspaper story about how IBM had invented the Talking Typewriter, and the speech problem was completely solved,” said Jurafsky. “I was upset! That’s the work I wanted to do. I’m not sure where I got the idea, probably watching TV shows like the Jetsons in this or Lost in Space.” Jurafsky, an associate professor of linguistics at issue: Stanford University, made the newspapers himself when he was among the 2002 winners of the MacArthur Fellowship, commonly known as the “genius grant.” page 2 The prestigious fellowship is awarded to individuals As I See It across the sciences, arts and public policy whose work by Nelson Morgan, Director has demonstrated singular vision and makes a unique contribution to their field. page 3 News Briefs The MacArthur Foundation selection committee cited Jurafsky for his work establishing “the foundations page 4 for developing systems that use natural language to interact with people.” Jurafsky’s research is in the EARS Diane Starr field of computational linguistics and focuses on using computers to model how people use language and Dan Jurafsky page 6 on getting computers to understand humans.
    [Show full text]
  • Towards a Computational History of the ACL: 1980-2008
    Towards a Computational History of the ACL: 1980–2008 Ashton Anderson Dan McFarland Dan Jurafsky Stanford University Stanford University Stanford University [email protected] [email protected] [email protected] Abstract history has been the use of topic models (Blei et al., 2003) to analyze the rise and fall of research top- We develop a people-centered computational ics to study the progress of science, both in general history of science that tracks authors over top- (Griffiths and Steyvers, 2004) and more specifically ics and apply it to the history of computa- in the ACL Anthology (Hall et al., 2008). tional linguistics. We present four findings in this paper. First, we identify the topical We extend this work with a more people-centered subfields authors work on by assigning auto- view of computational history. In this framework, matically generated topics to each paper in the we examine the trajectories of individual authors ACL Anthology from 1980 to 2008. Next, we across research topics in the field of computational identify four distinct research epochs where linguistics. By examining a single author’s paper the pattern of topical overlaps are stable and topics over time, we can trace the evolution of her different from other eras: an early NLP pe- academic efforts; by superimposing these individual riod from 1980 to 1988, the period of US traces over each other, we can learn how the entire government-sponsored MUC and ATIS eval- uations from 1989 to 1994, a transitory period field progressed over time. One goal is to investi- until 2001, and a modern integration period gate the use of these techniques for computational from 2002 onwards.
    [Show full text]
  • Dan Jurafsky Linguistics and Computer Science Stanford University
    Dan Jurafsky Linguistics and Computer Science Stanford University IPAM, May 23, 2016 ketchup America’s national foods Hamburger Frankfurter French fries Europe Hamburg Frankfurt French But what about ketchup? Fish preservation 2000 years ago ◦ Mon-Khmer, Tai-Kadai, Hmong-Mien speakers in Southeast China/ Southeast Asia Preserving fish to last through dry season ◦Fish, rice, salt, lactic acid fermentation 200 BCE: China expands Han Dynasty: Emperor Wu invades tribal Guangdong and Fujian. ◦Assimilating locals and their fermented seafoods ◦Pushing the rest south Innovations in Japan 700 CE: fermented fish rice è Japan called sushi (technically narezushi) 18th century: Use vinegar instead of lactic fermentation (and eat the rice) 19th century: eat the fish fresh Meanwhile… Southern Chinese sail/emigrate to SE Asia Propagating fish sauces and fish products. Also red rice wine and wine lees pastes 红糟 S Korea Japan China Burma Laos Philipines Thailand Vietnam Cambodia Malaysia Indonesia The fish/soy fermentation isogloss! Ishige (1993) The Age of European Exploration 1600: British and Dutch sail to Asia for spices, textiles, porcelain The sailors drink beer and wine But both go sour in the tropical heat (no hops yet) They arrive in Indonesia to find ethnic Chinese making: Arrack The ancestor of rum Distilled palm wine & red rice (from Arabic word for ‘sweat’) 1609: British start buying arrack 1704: The “common drink” of Europeans in Asia Sailors add limes (for scurvy) to make “punch”, first cocktail While buying arrack, the British picked up barrels of: Fish sauce Vietnamese nuoc mam or Thai nam pla What was this fish sauce called in the Zhangzhou dialect of Southern Min? “Ketchup” ke =鲑 salted fish chiap = 汁 “sauce” The British bring it home.
    [Show full text]
  • Dan Jurafsky
    Dan Jurafsky Department of Linguistics, [email protected] Stanford University www.stanford.edu/~jurafsky Margaret Jacks Hall FAX: (650) 723-5666 Stanford CA 94305-2150 Education B.A Linguistics, with honors, University of California at Berkeley, 1983 Ph.D. Computer Science, University of California at Berkeley, 1992 Postdoc, International Computer Science Institute, Berkeley, 1992-1995 Academic Employment Stanford University: Professor and Chair of Linguistics and Professor of Computer Science, 2014- Stanford University: Professor of Linguistics and (by courtesy) of Computer Science, 2010- Stanford University: Associate Professor of Linguistics and (by courtesy) of Computer Science, 2004-2010 University of Colorado, Associate Professor of Linguistics, Computer Science, Cognitive Science, 2001-2003 University of Colorado, Assistant Professor of Linguistics, Computer Science, and Cognitive Science, 1996-2001 Honors, Awards, & Fellowships MacArthur Fellowship, 2003 Fellow, Center for Advanced Study in the Behavioral Sciences, 2012-2013 NSF CAREER Award, 1998 Paper Awards: Best Paper WWW 2013 (Danescu-Niculescu-Mizil et al 2013); Best Paper EMNLP 2013 (Spitkovsky, Alshawi, Jurafsky 2013); Best Paper ACL 2006 (Snow, Jurafsky, and Ng 2006); Distinguished paper IJCAI 2001 (Gildea & Jurafsky 2001); Cogsci 1998 Marr Prize Honorable Mention (Narayanan and Jurafsky 1998) Awards for student advisees Robert M. Golden Medal for outstanding undergraduate thesis: Justine Kao, Stanford, 2011 Robert M. Golden Medal for outstanding undergraduate thesis: Richard Futrell, Stanford, 2010 Firestone Medal for outstanding undergraduate thesis: David Hall, Stanford, 2008 Outstanding Student Paper Award: Gregory, Raymond, Bell, Jurafsky (1999), Society for Text and Discourse Research Books and Edited Volumes 1. Jurafsky, Dan. 2014. The Language of Food. Pearson. 2. Huang, Chu-Ren and Jurafsky, Dan, eds.
    [Show full text]
  • Christopher David Manning
    Christopher David Manning Department of Linguistics Phone: +1 (650) 723-7683 Margaret Jacks Hall Web: http://nlp.stanford.edu/~manning/ Stanford CA 94305-2150 Email: [email protected] A. Education 1990–1994 Ph.D. Stanford University, Dept. of Linguistics, awarded January 1995. Dissertation: Ergativity: Argument Structure and Grammatical Relations. Committee: Joan Bresnan (chair), Mary Dalrymple, Ivan Sag, Peter Sells. 1984–1989 B.A. (Hons) with First Class Honours in Linguistics, The Australian National University. Additional majors in Mathematics and Computer Science. Honours Thesis: The Acquisition of Morphology. Advisor: Avery D. Andrews. Scholarships and Honors Bernard and Julia Bloch Memorial Fellow (student representative on the Lin- guistic Society of America Executive Committee), 1993–1995. Stanford Centennial Teaching Assistant (award in honor of outstanding teach- ing), 1993. Fellowships, 1991 and 1993 Linguistic Society of America Summer Institutes. Stanford University Fellowship, 1990–1994. University Medal, Australian National University, 1989. (Australian) National Undergraduate Scholarship, 1984–1989. B. Employment 02/14/2016– Thomas M. Siebel Professor in Machine Learning, Professor of Linguistics and of Computer Science, Stanford University. 09/01/2012–2016 Professor of Computer Science and Linguistics, Stanford University. 2010–2011 Visiting faculty at Google while on sabbatical from Stanford. 09/01/2006–2012 Associate Professor of Computer Science and Linguistics, Stanford University. 09/01/1999–2006 Assistant Professor of Computer Science and Linguistics, Stanford University. 1996–1999 Lecturer (tenured from 1998), Department of Linguistics, University of Sydney. 1994–1996 Assistant Professor, Computational Linguistics Program, Department of Phi- losophy, Carnegie Mellon University. 1989–1990 Teacher, Ichihara Ch¯u¯oGakk¯o, Chiba-ken, Japan 1996 Computer Systems Officer, Department of House of Representatives, Canberra, Australia C.
    [Show full text]
  • Dan Jurafsky
    Dan Jurafsky Department of Linguistics & [email protected] Department of Computer Science, http://www.stanford.edu/~jurafsky Stanford University Stanford CA 94305-2150 Education B.A Linguistics, with honors, University of California at Berkeley, 1983 Ph.D. Computer Science, University of California at Berkeley, 1992 Postdoc, International Computer Science Institute, Berkeley, 1992-1995 Academic Employment Stanford University: Professor and Chair of Linguistics and Professor of Computer Science, 2014- Stanford University: Professor of Linguistics and (by courtesy) of Computer Science, 2010- Stanford University: Associate Professor of Linguistics and (by courtesy) of Computer Science, 2004-2010 University of Colorado, Associate Professor of Linguistics, Computer Science, Cognitive Science, 2001-2003 University of Colorado, Assistant Professor of Linguistics, Computer Science, and Cognitive Science, 1996-2001 Honors, Awards, & Fellowships Fillmore Professor, Linguistics Society of America 2015 James Beard Foundation Book Award Finalist for The Language of Food, 2015 Fellow, Center for Advanced Study in the Behavioral Sciences, 2012-2013 MacArthur Fellowship, 2003 NSF CAREER Award, 1998 Paper Awards: 2017 Cozzarelli Prize (Voigt et al 2017); 2015 Roger V. Gould Prize (McFarland et al 2013); Best Paper WWW 2013 (Danescu-Niculescu-Mizil et al 2013); Best Paper EMNLP 2013 (Spitkovsky, Alshawi, Jurafsky 2013); Best Paper ACL 2006 (Snow, Jurafsky, and Ng 2006); Distinguished paper IJCAI 2001 (Gildea & Jurafsky 2001); Cogsci 1998 Marr Prize Honorable Mention (Narayanan and Jurafsky 1998) Awards for student advisees Robert M. Golden Medal for outstanding undergraduate thesis: Justine Kao, Stanford, 2011 Robert M. Golden Medal for outstanding undergraduate thesis: Richard Futrell, Stanford, 2010 Firestone Medal for outstanding undergraduate thesis: David Hall, Stanford, 2008 Outstanding Student Paper Award: Gregory, Raymond, Bell, Jurafsky (1999), Society for Text and Discourse Research Books and Edited Volumes 1.
    [Show full text]
  • Which Words Are Hard to Recognize? Prosodic, Lexical, and Disfluency
    Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates Sharon Goldwater, Dan Jurafsky, Christopher D. Manning To cite this version: Sharon Goldwater, Dan Jurafsky, Christopher D. Manning. Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates. Speech Communi- cation, Elsevier : North-Holland, 2010, 52 (3), pp.181. 10.1016/j.specom.2009.10.001. hal-00608401 HAL Id: hal-00608401 https://hal.archives-ouvertes.fr/hal-00608401 Submitted on 13 Jul 2011 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Accepted Manuscript Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates Sharon Goldwater, Dan Jurafsky, Christopher D. Manning PII: S0167-6393(09)00159-9 DOI: 10.1016/j.specom.2009.10.001 Reference: SPECOM 1837 To appear in: Speech Communication Received Date: 12 July 2009 Revised Date: 6 October 2009 Accepted Date: 9 October 2009 Please cite this article as: Goldwater, S., Jurafsky, D., Manning, C.D., Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates, Speech Communication (2009), doi: 10.1016/j.specom.2009.10.001 This is a PDF file of an unedited manuscript that has been accepted for publication.
    [Show full text]
  • Steven Bethard
    Steven Bethard Assistant Professor School of Information University of Arizona [email protected] http://bethard.faculty.arizona.edu/ http://scholar.google.com/citations?user=sXM8J5EAAAAJ Research Interests Natural language processing, machine learning, and information retrieval theory and applications, including constructing timelines from unstructured text, building information extraction models for health informatics, and automating educational feedback for personalized learning. Chronology of Education 2011-09 – 2013-08 Postdoc Institute of Cognitive Science, University of Colorado 2010-09 – 2011-08 Postdoc Department of Computer Science, KU Leuven 2009-06 – 2009-08 Postdoc Human Language Technology Center of Excellence, Johns Hopkins 2009-01 – 2010-08 Postdoc Department of Computer Science, Stanford University 2008-01 – 2008-12 Postdoc Institute of Cognitive Science, University of Colorado 2002-08 – 2007-12 Ph.D. Computer Science and Cognitive Science, University of Colorado 1998-08 – 2002-05 B.S., B.A. Computer Science, Mathematics, Linguistics, University of Arizona Chronology of Employment 2016-08 – present Assistant Professor University of Arizona, School of Information 2013-08 – 2016-08 Assistant Professor U. Alabama at Birmingham, Computer and Information Sciences 2011-10 – 2015-10 Visiting Professor KU Leuven Department of Computer Science Honors and Awards 2016-02 Kevin and Jo Ann Reilly Endowed Award ($1000), University of Alabama at Birmingham 2009-06 Vannevar Bush Best Paper Award ($1000), ACM/IEEE Joint Conference
    [Show full text]
  • Linguistic Structure in Statistical Machine Translation
    Linguistic Structure in Statistical Machine Translation zur Erlangung des akademischen Grades eines Doktors der Ingenieurwissenschaften der Fakult¨atf¨urInformatik des Karlsruher Instituts f¨urTechnologie (KIT) genehmigte Dissertation von Teresa Herrmann aus Dresden Tag der m¨undlichen Pr¨ufung: 19. Juni 2015 Erster Gutachter: Prof. Dr. Alexander Waibel Zweiter Gutachter: Dr. Stephan Vogel Abstract The first approaches to machine translation required linguists or language experts writing language-dependent translation rules by hand. In the last 20 years, statistical approaches applying machine learning techniques have be- come the state-of-the-art. They automatically learn translation rules from large parallel corpora of existing translations. This facilitates fast develop- ment of translation systems for new languages without the need for language experts writing rules. In phrase-based machine translation, new translations are constructed from phrases seen in previous translations during training. In most cases, this flat composition of phrases leads to good results, espe- cially for translations between languages with similar sentence structure. However, this strong suit of statistical translation exposes a weakness, if there are structural di↵erences between the languages. German is one of the languages where structural changes are necessary during translation, which increases the difficulty for translation. A particular challenge is the position of the verb in the German sentence, which depends on various factors that need to be taken into consideration when translating from or into German. The knowledge about sentence structure could provide helpful information for the translation process. This thesis investigates the influence of linguistic structure in statistical machine translation. The first part of this thesis addresses the particu- lar challenge that di↵erences in word order between languages present for translation.
    [Show full text]
  • Dan Jurafsky: Curriculum Vitae
    Dan Jurafsky Department of Linguistics & [email protected] Department of Computer Science, http://www.stanford.edu/~jurafsky Stanford University Stanford CA 94305-2150 Education B.A Linguistics, with honors, University of California at Berkeley, 1983 Ph.D. Computer Science, University of California at Berkeley, 1992 Postdoc, International Computer Science Institute, Berkeley, 1992-1995 Academic Employment Stanford University: Reynolds Professor in Humanities, Professor of Linguistics & of Computer Science, 2018- Stanford University: Professor and Chair of Linguistics and Professor of Computer Science, 2014-2020 Stanford University: Professor of Linguistics and (by courtesy) of Computer Science, 2010- Stanford University: Associate Professor of Linguistics and (by courtesy) of Computer Science, 2004-2010 University of Colorado, Associate Professor of Linguistics, Computer Science, Cognitive Science, 2001-2003 University of Colorado, Assistant Professor of Linguistics, Computer Science, and Cognitive Science, 1996-2001 Honors, Awards, & Fellowships Fellow, Association for Computational Linguistics 2020 Member, American Academy of Arts and Sciences 2020 Distinguished Alumni Award, Department of Electrical Engineering and Computer Science, UC Berkeley Fellow, American Association for the Advancement of Science 2020 Fellow, Linguistics Society of America 2019 Chair Professor, Jackson Eli Reynolds Professor in Humanities, Stanford, 2018- Fillmore Professor, Linguistics Society of America 2015 James Beard Foundation Book Award Finalist for The Language of Food, 2015 Fellow, Center for Advanced Study in the Behavioral Sciences, 2012-2013 MacArthur Fellowship, 2003 NSF CAREER Award, 1998 Paper Awards: 2017 Cozzarelli Prize (Voigt et al 2017); 2018 Cialdini Prize (Voigt et al 2017); 2015 Roger V. Gould Prize (McFarland et al 2013); Best Paper WWW 2013 (Danescu-Niculescu-Mizil et al 2013); Best Paper EMNLP 2013 (Spitkovsky, Alshawi, Jurafsky 2013); Shortlist for 2009 ACL Test-of-Time Award, 2019 (Mintz et al.
    [Show full text]
  • Dan Jurafsky
    DAN JURAFSKY Department of Linguistics, FAX: (650) 723-5666 Stanford University [email protected] Margaret Jacks Hall www.stanford.edu/˜jurafsky Stanford CA 94305-2150 EDUCATION B.A. 1983 Linguistics, with honors, University of California at Berkeley. Ph.D. 1992 Computer Science, University of California at Berkeley, Postdoc 1992–1995 International Computer Science Institute, Berkeley. ACADEMIC EMPLOYMENT 2010- Professor, Stanford University Department of Linguistics Professor by courtesy, Department of Computer Science 2004- Associate Professor, Stanford University Department of Linguistics Associate Professor by courtesy, Department of Computer Science 2004- Adjunct Associate Professor, University of Colorado Department of Linguistics 2001-2003 Associate Professor, University of Colorado Department of Linguistics, Department of Com- puter Science, and Institute of Cognitive Science 1996-2001 Assistant Professor, University of Colorado Department of Linguistics, Department of Com- puter Science, and Institute of Cognitive Science 1993–1994 Adjunct Assistant Professor, UC Berkeley Linguistics Department. HONORS • MacArthur Fellowship, 2003 • Best Paper Award, COLING/ACL 2006: Snow, Jurafsky, and Ng. 2006. • Distinguished Paper, at IJCAI 2001: Gildea & Jurafsky (2001) • Marr Prize Honorable Mention (Cognitive Science Society Best Student Paper Award): Narayanan and Jurafsky (1998) • NSF CAREER Award, 1998 • Awards for student advisees: • Robert M. Golden Medal (for Outstanding Undergraduate Thesis in the Humanities and the Creative Arts): Justine Kao, Stanford, 2011 • Robert M. Golden Medal (for Outstanding Undergraduate Thesis in the Humanities and the Creative Arts): Richard Futrell, Stanford, 2010 • Firestone Medal (for Outstanding Undergraduate Thesis in the social sciences, sciences, and engineering): David Hall, Stanford, 2008 • Outstanding Student Paper Award: Gregory, Raymond, Bell, Jurafsky (1999), Ninth An- nual Meeting of the Society for Text and Discourse 1 DAN JURAFSKY BOOKS AND Huang, Chu-Ren and Jurafsky, Dan, eds.
    [Show full text]