The Openhart 2013 Evalua on Workshop

Total Page：16

File Type：pdf, Size：1020Kb

Welcome to the OpenHaRT 2013 Evalua8on Workshop Informaon Technology Laboratory nist.gov/itl Informaon Access Division nist.gov/itl/iad Mark Przybocki, Mul(modal Informaon Group nist.gov/itl/iad/mig August 23rd, 2013 Washington D.C. , Omni Shoreham hotel The Mul8modal Informa8on Group’s Project Areas § Speech Recogni(on § Speaker Recogni(on § Dialog Management § Human Assisted Speaker Recogni(on § Topic Detec(on and Tracking § Speaker Segmentaon § Spoken Document Retrieval § Language Recogni(on § Voice Biometrics § ANSI/NIST-ITL Standard Voice Record § Tracking (Person/Object) § Text-to-Text § Event Detec(on § Speech-to-Text § Event Recoun(ng § Speech-to-Speech § Predic(ve Video Analy(cs § Image-to-Text § Metric Development § Named En(ty Iden(ficaon § (new) Data Analy(cs § Automac Content Extrac(on 2 Defini8on: MIG’s Evalua8on Cycle Evalua'on Driven Research NIST Data NIST Researchers Performance Planning NIST Core technology Assessment development Analysis and NIST NIST Workshop 3 NIST’s MT Program’s Legacy – Past 10 Years • 27 Evaluaon Events -- tracking the state-of-the-art in performance – (4) technology types text-text speech-text speech-speech Handwri[en_Images-text – (9) languages Arabic-2, Chinese, Dari, Farsi, Hindi, Korean, Pashto, Urdu, English • 11 genres of structured and unstructured content (nwire, web, Bnews, Bconv, food, speeches, editorials, handwri(ng-2, blogs, SMS, dialogs) • 60 Evaluaon Test Sets available to MT researchers (source, references, metrics, sample system output and official results for comparison) • Over 85 research groups >400 Primary Systems Evaluated AFRL – American Univ. Cairo – Apptek – ARL – BBN – BYU – Cambridge – Chinese Acad. Sci. – 5% 3% 1% CMU – Columbia Univ. – Fujitsu Research – 11% OpenMT Google – IBM – JHU – Kansas State – KCSL – Language Weaver – Microsoh Research – Ohio TIDES State – Oxford – Qatar – Queen Mary (London) – 23% 57% TRANSTAC RWTH Aachen – SAIC - Sakhr – SRI – Stanford – Systran – UMD – USC ISI – Univ. Edinburgh – GALE Univ. of Karlsruhe – Univ. of Montreal – Xerox Research BOLT 4 .

Copy Link

Recommended publications

Final Study Report on CEF Automated Translation Value Proposition in the Context of the European LT Market/Ecosystem
Final study report on CEF Automated Translation value proposition in the context of the European LT market/ecosystem FINAL REPORT A study prepared for the European Commission DG Communications Networks, Content & Technology by: Digital Single Market CEF AT value proposition in the context of the European LT market/ecosystem Final Study Report This study was carried out for the European Commission by Luc MEERTENS 2 Khalid CHOUKRI Stefania AGUZZI Andrejs VASILJEVS Internal identification Contract number: 2017/S 108-216374 SMART number: 2016/0103 DISCLAIMER By the European Commission, Directorate-General of Communications Networks, Content & Technology. The information and views set out in this publication are those of the author(s) and do not necessarily reflect the official opinion of the Commission. The Commission does not guarantee the accuracy of the data included in this study. Neither the Commission nor any person acting on the Commission’s behalf may be held responsible for the use which may be made of the information contained therein. ISBN 978-92-76-00783-8 doi: 10.2759/142151 © European Union, 2019. All rights reserved. Certain parts are licensed under conditions to the EU. Reproduction is authorised provided the source is acknowledged. 2 CEF AT value proposition in the context of the European LT market/ecosystem Final Study Report CONTENTS Table of figures ................................................................................................................................................ 7 List of tables ..................................................................................................................................................
[Show full text]
Ruken ÇAkici
RUKEN ÇAKICI personal information Official Ruket Çakıcı Name Born in Turkey, 23 June 1978 email [email protected] website http://www.ceng.metu.edu.tr/˜ruken phone (H) +90 (312) 210 6968 · (M) +90 (532) 557 8035 work experience 2010- Instructor, METU METU Research and Teaching duties 1999-2010 Research Assistant, METU — Ankara METU Teaching assistantship of various courses education 2002-2008 University of Edinburgh, UK Doctor of School: School of Informatics Philosophy Thesis: Wide-Coverage Parsing for Turkish Advisors: Prof. Mark Steedman & Prof. Miles Osborne 1999-2002 Middle East Technical University Master of Science School: Computer Engineering Thesis: A Computational Interface for Syntax and Morphemic Lexicons Advisor: Prof. Cem Bozs¸ahin 1995-1999 Middle East Technical University Bachelor of Science School: Computer Engineering projects 1999-2001 AppTek/ Lernout & Hauspie Inc Language Pairing on Functional Structure: Lexical- Functional Grammar Based Machine Translation for English – Turkish. 150000USD. · Consultant developer 2007-2011 TUB¨ MEDID Turkish Discourse Treebank Project, TUBITAK 1001 program (107E156), 137183 TRY. · Researcher · (Now part of COST Action IS1312 (TextLink)) 2012-2015 Unsupervised Learning Methods for Turkish Natural Language Processing, METU BAP Project (BAP-08-11-2012-116), 30000 TRY. · Primary Investigator 2013-2015 TwiTR: Turkçe¨ için Sosyal Aglarda˘ Olay Bulma ve Bulunan Olaylar için Konu Tahmini (TwiTR: Event detection and Topic identification for events in social networks for Turkish language), TUBITAK 1001 program (112E275), 110750 TRY. · Researcher· (Now Part of ICT COST Action IC1203 (ENERGIC)) 2013-2016 Understanding Images and Visualizing Text: Semantic Inference and Retrieval by Integrating Computer Vision and Natural Language Processing, TUBITAK 1001 program (113E116), 318112 TRY.
[Show full text]
Statistical Machine Translation from English to Tuvan*
Statistical Machine Translation from English to Tuvan* Rachel Killackey, Swarthmore College rkillac [email protected] Linguistics Senior Thesis 2013 Abstract This thesis aims to describe and analyze findings of the Tuvan Machine Translation Project, which attempts to create a functional statistical machine translation (SMT) model between English and Tuvan, a minority language spoken in southern Siberia. Though most Tuvan speakers are also fluent in Russian, easily accessible SMT technology would allow for simpler English translation without the use of Russian as an intermediary language. The English to Tuvan half of the system that I examine makes consistent morphological errors, particularly involving the absence of the accusative suffix with the basic form -ni. Along with a typological analysis of these errors, I show that the introduction of novel data that corrects for the missing accusative suffix can improve the performance of an SMT system. This result leads me to conclude that SMT can be a useful avenue for efficient translation. However, I also argue that SMT may benefit from the incorporation of some linguistic knowledge such as morphological rules in the early steps of creating a system. 1. Introduction This thesis explores the field of machine translation (MT), the use of computers in rendering one natural language into another, with a specific focus on MT between English and Tuvan, a Turkic language spoken in south central Siberia. While MT is a growing force in the translation of major languages with millions of speakers such as French, Spanish, and Russian, minority and non-dominant languages with relatively few numbers of speakers have been largely ignored.
[Show full text]
Making Amharic to English Language Translator For
Hana Demas Making Amharic to English Language Translator for iOS Helsinki Metropolia University of Applied Sciences Degree Programme In Information Technology Thesis Date 5.5.2016 2 Author(s) Hana Belete Demas Title Amharic To English Language Translator For iOS Number of Pages 54 pages + 1 appendice Date 5 May 2016 Degree Information Technology Engineering Degree Programme Information Technology Specialisation option Software Engineering Instructor(s) Petri Vesikivi The purpose of this project was to build a language translator for Amharic-English language pair, which in the beginning of the project was not supported by any of the known translation systems. The goal of this project was to make a language translator application for Amharic English language pair using swift language for iOS platform. The project has two components. The first one is the language translator application described above and the second component is an integrated Amharic custom keyboard which makes the user able to type Amharic letters which are not supported by iOS 9 system keyboard. The Amharic language has more than 250 letters and numbers and they are represented using extended keys. The project was implemented using the Swift language. At the end of the project an iOS application to translate English to Amharic and vice versa was made. The translator applications uses the translation system which was built on the Microsoft Translator Hub and accessed using Microsoft Translator API. The application can be used to translate texts from Amharic to English or vice versa. Keywords API, iOS, Custom Keyboard, Swift, Microsoft Translator Hub 3 Contents 1. Introduction ............................................................................................................... 1 2.
[Show full text]
Proceedings of the 5Th Conference on Machine
EMNLP 2020 Fifth Conference on Machine Translation Proceedings of the Conference November 19-20, 2020 Online c 2020 The Association for Computational Linguistics Order copies of this and other ACL proceedings from: Association for Computational Linguistics (ACL) 209 N. Eighth Street Stroudsburg, PA 18360 USA Tel: +1-570-476-8006 Fax: +1-570-476-0860 [email protected] ISBN 978-1-948087-81-0 ii Introduction The Fifth Conference on Machine Translation (WMT 2020) took place on Thursday, November 19 and Friday, November 20, 2020 immediately following the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020). This is the fifth time WMT has been held as a conference. The first time WMT was held as a conference was at ACL 2016 in Berlin, Germany, the second time at EMNLP 2017 in Copenhagen, Denmark, the third time at EMNLP 2018 in Brussels, Belgium, and the fourth time at ACL 2019 in Florence, Italy. Prior to being a conference, WMT was held 10 times as a workshop. WMT was held for the first time at HLT-NAACL 2006 in New York City, USA. In the following years the Workshop on Statistical Machine Translation was held at ACL 2007 in Prague, Czech Republic, ACL 2008, Columbus, Ohio, USA, EACL 2009 in Athens, Greece, ACL 2010 in Uppsala, Sweden, EMNLP 2011 in Edinburgh, Scotland, NAACL 2012 in Montreal, Canada, ACL 2013 in Sofia, Bulgaria, ACL 2014 in Baltimore, USA, EMNLP 2015 in Lisbon, Portugal. The focus of our conference is to bring together researchers from the area of machine translation and invite selected research papers to be presented at the conference.
[Show full text]
Improvements in RWTH LVCSR Evaluation Systems for Polish, Portuguese, English, Urdu, and Arabic
Improvements in RWTH LVCSR Evaluation Systems for Polish, Portuguese, English, Urdu, and Arabic M. Ali Basha Shaik1, Zoltan Tuske¨ 1, M. Ali Tahir1, Markus Nußbaum-Thom1, Ralf Schluter¨ 1, Hermann Ney1;2 1Human Language Technology and Pattern Recognition – Computer Science Department RWTH Aachen University, 52056 Aachen, Germany 2Spoken Language Processing Group, LIMSI CNRS, Paris, France f shaik, tuske, tahir, nussbaum, schlueter, ney [email protected] Abstract acoustical mismatch between the training and testing can be re- In this work, Portuguese, Polish, English, Urdu, and Arabic duced for any target language, by exploiting matched data from automatic speech recognition evaluation systems developed by other languages [8]. Alternatively, a few attempts have been the RWTH Aachen University are presented. Our LVCSR sys- made to incorporate GMM within a framework of deep neu- tems focus on various domains like broadcast news, sponta- ral networks (DNN). The joint training of GMM and shallow neous speech, and podcasts. All these systems but Urdu are bottleneck features was proposed using the sequence MMI cri- used for Euronews and Skynews evaluations as part of the EU- terion, in which the derivatives of the error function are com- Bridge project. Our previously developed LVCSR systems were puted with respect to GMM parameters and applied the chain improved using different techniques for the aforementioned lan- rule to update the GMM simultaneously with the bottleneck guages. Signiﬁcant improvements are obtained using multi- features [9]. On the other hand, a different GMM-DNN integra- lingual tandem and hybrid approaches, minimum phone error tion approach was proposed in [10] by taking advantage of the training, lexical adaptation, open vocabulary long short term softmax layer, which deﬁnes a log-linear model.
[Show full text]
Neural Speech Translation at Apptek
Neural Speech Translation at AppTek Evgeny Matusov, Patrick Wilken, Parnia Bahar∗, Julian Schamper∗, Pavel Golik, Albert Zeyer∗, Joan Albert Silvestre-Cerda`+, Adria` Mart´ınez-Villaronga+, Hendrik Pesch, and Jan-Thorsten Peter Applications Technology (AppTek), Aachen, Germany ematusov,pwilken,pbahar,jschamper,pgolik,jsilvestre,amartinez,hpesch,jtpeter @apptek.com { } + ∗Also RWTH Aachen University, Germany Also Universitat Politecnica` de Valencia,` Spain Abstract processing, and punctuation prediction. Section 4 gives an This work describes AppTek’s speech translation pipeline overview of our ASR system. Section 5 describes the details that includes strong state-of-the-art automatic speech recog- of AppTek’s NMT system. Section 6 gives details of how nition (ASR) and neural machine translation (NMT) com- ASR confusion networks can be encoded as an input of the ponents. We show how these components can be tightly NMT system. In Section 7, we describe our direct speech coupled by encoding ASR confusion networks, as well as translation prototype. The results of our speech translation ASR-like noise adaptation, vocabulary normalization, and experiments are summarized in Section 8. implicit punctuation prediction during translation. In another experimental setup, we propose a direct speech translation 2. Related Work approach that can be scaled to translation tasks with large amounts of text-only parallel training data but a limited num- Theoretical background for tighter coupling of statistical ber of hours of recorded and human-translated speech. ASR and MT systems had been ﬁrst published in [3]. In prac- tice, it was realized e. g. as statistical phrase-based transla- tion of ASR word lattices with acoustic and language model 1.
[Show full text]
Translation of Languages: Fourteen Essays, Ed
[Written 15 July 1949. Published in: Machine translation of languages: fourteen essays, ed. by William N. Locke and A. Donald Booth (Technology Press of the Massachusetts Institute of Technology, Cambridge, Mass., and John Wiley & Sons, Inc., New York, 1955), p.15-23.] Translation WARREN WEAVER There is no need to do more than mention the obvious fact that a multiplicity of languages impedes cultural interchange between the peoples of the earth, and is a serious deterrent to international understanding. The present memorandum, assuming the validity and importance of this fact, contains some comments and suggestions bearing on the possibility of contributing at least something to the solution of the world-wide translation problem through the use of electronic computers of great capacity, flexibility, and speed. The suggestions of this memorandum will surely be incomplete and naïve, and may well be patently silly to an expert in the field—for the author is certainly not such. A War Anecdote—Language Invariants During the war a distinguished mathematician whom we will call P, an ex-German who had spent some time at the University of Istanbul and had learned Turkish there, told W. W. the following story. A mathematical colleague, knowing that P had an amateur interest in cryptography, came to P one morning, stated that he had worked out a deciphering technique, and asked P to cook up some coded message on which he might try his scheme. P wrote out in Turkish a message containing about 100 words; simplified it by replacing the Turkish letters ç, ğ, ĭ, ö, ş, and ü by c, g, i, o, s, and u respectively; and then, using something more complicated than a simple substitution cipher, reduced the message to a column of five-digit numbers.
[Show full text]
Machine Translation Summit XVI
Machine Translation Summit XVI http://www.mtsummit2017.org Proceedings of MT Summit XVI Vol.1 Research Track Sadao Kurohashi, Pascale Fung, Editors MT Summit XVI September 18 – 22, 2017 -- Nagoya, Aichi, Japan Proceedings of MT Summit XVI, Vol. 1: MT Research Track Sadao Kurohashi (Kyoto University) & Pascale Fung (Hong Kong University of Science and Technology), Eds. Co-hosted by International Asia-Pacific Association for Association for Graduate School of Machine Translation Machine Translation Informatics, Nagoya University http://www.eamt.org/iamt.php http://www.aamt.info http://www.is.nagoya-u.ac.jp/index_en.html ©2017 The Authors. These articles are licensed under a Creative Commons 3.0 license, no derivative works, attribution, CC-BY-ND. Research Program Co-chairs Sadao Kurohashi Kyoto University Pascale Fung Hong Kong University of Science and Technology Research Program Committee Yuki Arase Osaka University Laurent Besacier LIG Houda Bouamor Carnegie Mellon University Hailong Cao Harbin Institute of Technology Michael Carl Copenhagen Business School Daniel Cer Google Boxing Chen NRC Colin Cherry NRC Chenhui Chu Osaka University Marta R. Costa-jussa Universitat Politècnica de Catalunya Steve DeNeefe SDL Language Weaver Markus Dreyer Amazon Kevin Duh Johns Hopkins University Andreas Eisele DGT, European Commission Christian Federmann Microsoft Research Minwei Feng IBM Watson Group Mikel L. Forcada Universitat d'Alacant George Foster National Research Council Isao Goto NHK Spence Green Lilt Inc Eva Hasler SDL Yifan He Bosch Research
[Show full text]
Telecommunications for the Deaf and Hard of Hearing, Inc. Et Al
William B. Wilhelm, Jr. Christian E. Hoefly Jr. +1.202.739.3000 [email protected] [email protected] January 16, 2020 Via ECFS Marlene H. Dortch, Secretary Federal Communications Commission 445 12th Street, SW Room TW-A325 Washington, DC 20554 Re: Notice of Ex Parte Communication RM-11848; CG Docket No. 05-231 Telecommunications for the Deaf and Hard of Hearing, Inc. et al. Petition for Declaratory Ruling and/or Rulemaking on Live Closed Captioning Quality Metrics and the Use of Automatic Speech Recognition Technologies; Closed Captioning of Video Programming Dear Ms. Dortch: On January 14, 2019, Mudar Yaghi, Mike Veronis, and the undersigned counsel met with Diane Burstein,1 Suzy Rosen Singleton, Eliot Greenwald and Debra Patkin of the Federal Communications Commission’s (“Commission”) Consumer and Governmental Affairs Bureau (“Bureau”) to discuss AppTek’s comments2 and reply comments3 to the Telecommunications for the Deaf and Hard of Hearing Inc., et al., Petition for Declaratory Ruling and/or Rulingmaking on Live Closed Captioning Quality Metrics and the Use of Automatic Speech Recognition Technologies.4 1 Ms. Burstein participated in the first part of the meeting and recused herself during the discussion regarding quality metrics. 2 AppTek Comments, https://www.fcc.gov/ecfs/filing/101585639283 (filed October 15, 2019). 3 AppTek Reply Comments, https://www.fcc.gov/ecfs/filing/1030761604294 (filed October 30, 2019). 4 Telecommunications for the Deaf and Hard of Hearing, Inc. (TDI), et al., Petition for Declaratory Ruling and/or Rulemaking on Live Closed Captioning Quality Metrics and the Use of Automatic Speech Recognition Technologies, CG Docket No.
[Show full text]
September 2010
Language | Technology | Business September 2010 Region Focus: North America Doing language business with the US government Translation industry in Cuba Adding personalization to the localization mix No recession for translation Using the Review Index to measure content change 01Cover#114.indd 1 8/12/10 9:09:53 AM 02-03 Ads #114.indd 2 8/12/10 9:10:51 AM 02-03 Ads #114.indd 3 8/12/10 9:10:51 AM Know-how for Global Success Seattle, Washington, October 6-8, 2010 The conference Global Business Best Practices for global business know-how. Want to venture into new international markets? Experienced business people will provide practical insights and advice Fifty informative sessions on global business best practices, to give you confidence on the global playing field. international marketing, new media, content strategies, global websites, localization and translation management. Managing Global Websites Large exhibit hall with service and technology vendors. Need to take your company’s website global? Learn from companies that have outstanding web presences: representa- tives offer their expertise for launching and maintaining your site. Keynote speaker Advanced Localization Management Are you already involved in localization, but would like to learn more? Brian Solis, author of Engage: The Complete Guide for Industry experts provide the latest information on trends, processes, Brands and Businesses to Build, Cultivate, and Measure technologies and influences that shape the world of localization. Success in the New Web, principal of FutureWorks Localization Core Competencies Brian Solis is globally recognized as one of the most prominent thought leaders and published authors in Do you need to localize, but don’t know where to start? new media.
[Show full text]
International Conference Language Technologies for All (Lt4all): Enabling Linguistic Diversity and Multilingualism Worldwide
Organized and supported by International Conference Language Technologies for All (LT4All): Enabling Linguistic Diversity and Multilingualism Worldwide in the framework of the 2019 International Year of Indigenous Languages 4 - 6 December 2019 UNESCO Headquarters, Paris, France Conference programme at a glance In partnership with Programme 5 December 2019 (Room II) Day 2. ACHIEVEMENT Applying language technologies for linguistic diversity and multilingualism (Room II and Poster Area: Hall Segur) 08:00-09:00 Registration 09:00-10:15 Opening Session Moderator: Mr Moez Chakchouk, Assistant Director-General for Communication and Information, UNESCO Mr Moez Chakchouk Assistant Director- UNESCO General for Communication and Information Ms Anne Karin Olli Vice Minister National Government, Norway Ministry of Local Government and Modernisation Ms Dorothy Gordon Chair UNESCO Intergovernmental Ghana Information for All Programme (IFAP) Ms Aiolupotea Sina Deputy Chief Executive, New Zealand Ministry for Pacific New Zealand Aiono Regional Partnerships Peoples Mr Khalid Choukri Secretary General European Language Resources France Association (ELRA) and Special Interest Group: Under-resourced languages (SIGUL) Ms Lixin Tian Director-General Department of Language The People’s Information Management, Republic of Ministry of Education China Introduction of the LT4ALL: Language Technologies for All by the co-chairs of LT4All Day 2-3 10:15-10:30 Ms Sakriani Sakti Associate Professor, Nara Institute of Science and Technology, Japan Secretary of SIGUL, ELRA-ISCA
[Show full text]