Using Ontology-Based Approaches to Representing Speech Transcripts for Automated Speech Scoring
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
An Artificial Neural Network Approach for Sentence Boundary Disambiguation in Urdu Language Text
The International Arab Journal of Information Technology, Vol. 12, No. 4, July 2015 395 An Artificial Neural Network Approach for Sentence Boundary Disambiguation in Urdu Language Text Shazia Raj, Zobia Rehman, Sonia Rauf, Rehana Siddique, and Waqas Anwar Department of Computer Science, COMSATS Institute of Information Technology, Pakistan Abstract: Sentence boundary identification is an important step for text processing tasks, e.g., machine translation, POS tagging, text summarization etc., in this paper, we present an approach comprising of Feed Forward Neural Network (FFNN) along with part of speech information of the words in a corpus. Proposed adaptive system has been tested after training it with varying sizes of data and threshold values. The best results, our system produced are 93.05% precision, 99.53% recall and 96.18% f-measure. Keywords: Sentence boundary identification, feed forwardneural network, back propagation learning algorithm. Received April 22, 2013; accepted September 19, 2013; published online August 17, 2014 1. Introduction In such conditions it would be difficult for a machine to differentiate sentence terminators from Sentence boundary disambiguation is a problem in natural language processing that decides about the ambiguous punctuations. beginning and end of a sentence. Almost all language Urdu language processing is in infancy stage and processing applications need their input text split into application development for it is way slower for sentences for certain reasons. Sentence boundary number of reasons. One of these reasons is lack of detection is a difficult task as very often ambiguous Urdu text corpus, either tagged or untagged. However, punctuation marks are used in the text. -
Towards Interpretation As Natural Logic Abduction
4Pin1-33 The 32nd Annual Conference of the Japanese Society for Artificial Intelligence, 2018 Towards Interpretation as Natural Logic Abduction Naoya Inoue∗1 Pontus Stenetorp∗2 Sebastian Riedel∗2 Kentaro Inui∗1∗3 ∗1 Tohoku University ∗2 University College London ∗3 RIKEN Center for Advanced Intelligence Project This paper studies an abductive reasoning framework on natural language representations. We show how to incorporate Natural Logic (McCartney et al. 2009) into Interpretation as Abduction (Hobbs et al. 1993). The resulting framework avoids the difficulty of natural language-formal representation conversion, while handling im- portant linguistic expressions such as negations and quantifiers. For proof-of-concept, we demonstrate that missing, implicit premises of arguments can be recovered by the proposed framework by a manual example walkthrough. 1. Introduction is orthogonal to the conventional formal representation- based approaches. More specifically, we study Natural Abduction is inference to the best explanation. This pa- Logic [Maccartney 09, Bowman 13, Angeli 14] in the con- per explores abduction for natural language understanding. text of abductive reasoning. Natural Logic is a framework Given a textual observation, the task is to generate the originally developed for entailment prediction. The advan- best, textual explanation. For example, the task of find- tage is that (i) it works on natural language expression di- ing a missing assumption of an argument [Habernal 17] is a rectly, and (ii) it supports important linguistic expressions special form of abduction. Suppose the following argument ∗1 such as negations and quantifiers. is given : We show how Natural Logic-based inference can be inte- (1) Claim: Immigration is really a problem. -
A Comparative Study of Pretrained Language Models for Automated Essay Scoring with Adversarial Inputs
A Comparative Study of Pretrained Language Models for Automated Essay Scoring with Adversarial Inputs 1st Phakawat Wangkriangkri 2nd Chanissara Viboonlarp 3rd Attapol T. Rutherford Department of Computer Engineering Department of Computer Engineering Department of Linguistics Chulalongkorn University Chulalongkorn University Chulalongkorn University Bangkok, Thailand Bangkok, Thailand Bangkok, Thailand [email protected] [email protected] [email protected] 4th Ekapol Chuangsuwanich Department of Computer Engineering Chulalongkorn University Bangkok, Thailand [email protected] Abstract—Automated Essay Scoring (AES) is a task that for scoring essays. We experimented three popular transfer- deals with grading written essays automatically without human learning text encoders: Global Vectors for Word Repre- intervention. This study compares the performance of three sentation (GloVe) [6], Embeddings from Language Models AES models which utilize different text embedding methods, namely Global Vectors for Word Representation (GloVe), Em- (ELMo) [7], and Bidirectional Encoder Representations from beddings from Language Models (ELMo), and Bidirectional Transformers (BERT) [8]. An ideal AES system should both Encoder Representations from Transformers (BERT). We used correlate with manually-graded essay scores and be resistant two evaluation metrics: Quadratic Weighted Kappa (QWK) and to adversarial hacks; thus, we evaluated the models against a novel “robustness”, which quantifies the models’ ability to both criteria. detect adversarial essays created by modifying normal essays to cause them to be less coherent. We found that: (1) the BERT- Our contributions can be summarized as follows: based model achieved the greatest robustness, followed by the • We developed three AES-specific models utilizing dif- GloVe-based and ELMo-based models, respectively, and (2) fine- ferent transfer-learning text encoders, and compared tuning the embeddings improves QWK but lowers robustness. -
The Effects of Automated Essay Scoring As a High School Classroom Intervention
UNLV Retrospective Theses & Dissertations 1-1-2008 The effects of automated essay scoring as a high school classroom intervention Kathie L Frost University of Nevada, Las Vegas Follow this and additional works at: https://digitalscholarship.unlv.edu/rtds Repository Citation Frost, Kathie L, "The effects of automated essay scoring as a high school classroom intervention" (2008). UNLV Retrospective Theses & Dissertations. 2839. http://dx.doi.org/10.25669/37p4-kdh7 This Dissertation is protected by copyright and/or related rights. It has been brought to you by Digital Scholarship@UNLV with permission from the rights-holder(s). You are free to use this Dissertation in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s) directly, unless additional rights are indicated by a Creative Commons license in the record and/or on the work itself. This Dissertation has been accepted for inclusion in UNLV Retrospective Theses & Dissertations by an authorized administrator of Digital Scholarship@UNLV. For more information, please contact [email protected]. THE EFFECTS OF AUTOMATED ESSAY SCORING AS A HIGH SCHOOL CLASSROOM INTERVENTION by Kathie L. Frost Bachelor of Science University of Arizona Tucson, Arizona Master of Business Administration University of Nevada, Las Vegas A dissertation submitted in partial fulfillment of the requirements for the Doctor of Philosophy Degree in Curriculum and Instruction Department of Curriculum and Instruction College of Education Graduate College University of Nevada, Las Vegas December 2008 UMI Number: 3352171 Copyright 2009 by Frost, Kathie L. -
Semantic Role Labeling 2
Semantic Role Labeling 2 Outline • Semantic role theory • Designing semantic role annotation project ▫ Granularity ▫ Pros and cons of different role schemas ▫ Multi-word expressions 3 Outline • Semantic role theory • Designing semantic role annotation project ▫ Granularity ▫ Pros and cons of different role schemas ▫ Multi-word expressions 4 Semantic role theory • Predicates tie the components of a sentence together • Call these components arguments • [John] opened [the door]. 5 Discovering meaning • Syntax only gets you so far in answering “Who did what to whom?” John opened the door. Syntax: NPSUB V NPOBJ The door opened. Syntax: NPSUB V 6 Discovering meaning • Syntax only gets you so far in answering “Who did what to whom?” John opened the door. Syntax: NPSUB V NPOBJ Semantic roles: Opener REL thing opened The door opened. Syntax: NPSUB V Semantic roles: thing opened REL 7 Can the lexicon account for this? • Is there a different sense of open for each combination of roles and syntax? • Open 1: to cause something to become open • Open 2: become open • Are these all the senses we would need? (1) John opened the door with a crowbar. Open1? (2) They tried the tools in John’s workshop one after the other, and finally the crowbar opened the door. Still Open1? 8 Fillmore’s deep cases • Correspondence between syntactic case and semantic role that participant plays • “Deep cases”: Agentive, Objective, Dative, Instrument, Locative, Factitive • Loosely associated with syntactic cases; transformations result in the final surface case 9 The door opened. Syntax: NPSUB V Semantic roles: Objective REL John opened the door. Syntax: NPSUB V NPOBJ Semantic roles: Agentive REL Objective The crowbar opened the door. -
A Comparison of Knowledge Extraction Tools for the Semantic Web
A Comparison of Knowledge Extraction Tools for the Semantic Web Aldo Gangemi1;2 1 LIPN, Universit´eParis13-CNRS-SorbonneCit´e,France 2 STLab, ISTC-CNR, Rome, Italy. Abstract. In the last years, basic NLP tasks: NER, WSD, relation ex- traction, etc. have been configured for Semantic Web tasks including on- tology learning, linked data population, entity resolution, NL querying to linked data, etc. Some assessment of the state of art of existing Knowl- edge Extraction (KE) tools when applied to the Semantic Web is then desirable. In this paper we describe a landscape analysis of several tools, either conceived specifically for KE on the Semantic Web, or adaptable to it, or even acting as aggregators of extracted data from other tools. Our aim is to assess the currently available capabilities against a rich palette of ontology design constructs, focusing specifically on the actual semantic reusability of KE output. 1 Introduction We present a landscape analysis of the current tools for Knowledge Extraction from text (KE), when applied on the Semantic Web (SW). Knowledge Extraction from text has become a key semantic technology, and has become key to the Semantic Web as well (see. e.g. [31]). Indeed, interest in ontology learning is not new (see e.g. [23], which dates back to 2001, and [10]), and an advanced tool like Text2Onto [11] was set up already in 2005. However, interest in KE was initially limited in the SW community, which preferred to concentrate on manual design of ontologies as a seal of quality. Things started changing after the linked data bootstrapping provided by DB- pedia [22], and the consequent need for substantial population of knowledge bases, schema induction from data, natural language access to structured data, and in general all applications that make joint exploitation of structured and unstructured content. -
ACL 2019 Social Media Mining for Health Applications (#SMM4H)
ACL 2019 Social Media Mining for Health Applications (#SMM4H) Workshop & Shared Task Proceedings of the Fourth Workshop August 2, 2019 Florence, Italy c 2019 The Association for Computational Linguistics Order copies of this and other ACL proceedings from: Association for Computational Linguistics (ACL) 209 N. Eighth Street Stroudsburg, PA 18360 USA Tel: +1-570-476-8006 Fax: +1-570-476-0860 [email protected] ISBN 978-1-950737-46-8 ii Preface Welcome to the 4th Social Media Mining for Health Applications Workshop and Shared Task - #SMM4H 2019. The total number of users of social media continues to grow worldwide, resulting in the generation of vast amounts of data. Popular social networking sites such as Facebook, Twitter and Instagram dominate this sphere. According to estimates, 500 million tweets and 4.3 billion Facebook messages are posted every day 1. The latest Pew Research Report 2, nearly half of adults worldwide and two- thirds of all American adults (65%) use social networking. The report states that of the total users, 26% have discussed health information, and, of those, 30% changed behavior based on this information and 42% discussed current medical conditions. Advances in automated data processing, machine learning and NLP present the possibility of utilizing this massive data source for biomedical and public health applications, if researchers address the methodological challenges unique to this media. In its fourth iteration, the #SMM4H workshop takes place in Florence, Italy, on August 2, 2019, and is co-located with the -
Full-Text Processing: Improving a Practical NLP System Based on Surface Information Within the Context
Full-text processing: improving a practical NLP system based on surface information within the context Tetsuya Nasukawa. IBM Research Tokyo Resem~hLaborat0ry t623-14, Shimotsurum~, Yimmt;0¢sl{i; I<almgawa<kbn 2421,, J aimn • nasukawa@t,rl:, vnet;::ibm icbm Abstract text. Without constructing a i):recige filodel of the eohtext through, deep sema~nfiCamtlys~is, our frmne= Rich information fl)r resolving ambigui- "work-refers .to a set(ff:parsed trees. (.r~sltlt~ 9 f syn- ties m sentence ~malysis~ including vari- : tacti(" miaiysis)ofeach sexitencd in t.li~;~i'ext as (:on- ous context-dependent 1)rol)lems. can be ob- text ilfformation, Thus. our context model consists tained by analyzing a simple set of parsed of parse(f trees that are obtained 1)y using mi exlst- ~rces of each senten('e in a text withom il!g g¢lwral syntactic parser. Excel)t for information constructing a predse model of the contex~ ()It the sequence of senl;,en('es, olIr framework does nol tl(rough deep senmntic.anMysis. Th.us. pro- consider any discourse stru(:~:ure mwh as the discourse cessmg• ,!,' a gloup" of sentem' .., '~,'(s togethel. i ': makes.' • " segmenm, focus space stack, or dominant hierarclty it.,p.(,)ss{t?!e .t.9 !~npl:ovel-t]le ~ccui'a~'Y (?f a :: :.it~.(.fi.ii~idin:.(cfi.0szufid, Sht/!er, dgs6)i.Tli6refbi.e, om< ,,~ ~-. ~t.ehi- - ....t sii~ 1 "~g;. ;, ~-ni~chin¢''ti-mslat{b~t'@~--..: .;- . .? • . : ...... - ~,',". ..........;-.preaches ":........... 'to-context pr0cessmg,,' • and.m" "tier-ran : ' le d at. •tern ; ::'Li:In i. thin.j.;'P'..p) a (r ~;.!iwe : .%d,es(tib~, ..i-!: .... -
Syntax Role for Neural Semantic Role Labeling
Syntax Role for Neural Semantic Role Labeling Zuchao Li Hai Zhao∗ Shanghai Jiao Tong University Shanghai Jiao Tong University Department of Computer Science and Department of Computer Science and Engineering Engineering [email protected] [email protected] Shexia He Jiaxun Cai Shanghai Jiao Tong University Shanghai Jiao Tong University Department of Computer Science and Department of Computer Science and Engineering Engineering [email protected] [email protected] Semantic role labeling (SRL) is dedicated to recognizing the semantic predicate-argument struc- ture of a sentence. Previous studies in terms of traditional models have shown syntactic informa- tion can make remarkable contributions to SRL performance; however, the necessity of syntactic information was challenged by a few recent neural SRL studies that demonstrate impressive performance without syntactic backbones and suggest that syntax information becomes much less important for neural semantic role labeling, especially when paired with recent deep neural network and large-scale pre-trained language models. Despite this notion, the neural SRL field still lacks a systematic and full investigation on the relevance of syntactic information in SRL, for both dependency and both monolingual and multilingual settings. This paper intends to quantify the importance of syntactic information for neural SRL in the deep learning framework. We introduce three typical SRL frameworks (baselines), sequence-based, tree-based, and graph-based, which are accompanied by two categories of exploiting syntactic information: syntax pruning- based and syntax feature-based. Experiments are conducted on the CoNLL-2005, 2009, and 2012 benchmarks for all languages available, and results show that neural SRL models can still benefit from syntactic information under certain conditions. -
Automated Ontology Creation Using XML Schema Elements
Automated Ontology Creation using XML Schema Elements Samuel Suhas Singapogu, Paulo C. G. Costa, J. Mark Pullen C4I center, George Mason University Fairfax, VA, USA [ssingapo,pcosta,mpullen]@c4i.gmu.edu Abstract—Ontologies are commonly used to represent formal semantics in a computer system, usually capturing them in the Although ontologies are valuable assets in system model- form of concepts, relationships and axioms. Axioms convey ing, testing and analysis, the process of manually creating an asserted knowledge and support inferring new knowledge ontology for a complex system is inherently tedious and through logical reasoning. For complex systems, the process of error prone. Existing methods of knowledge discovery and creating ontologies manually can be tedious and error-prone. ontology creation usually are based on text mining of a data Many automated methods of knowledge discovery are based on mining domain text corpus, but current state-of-the-art meth- corpus for that domain. XML-based systems capture the ods using this approach fail to consider properly semantic data structure and syntax of all necessary and meaningful ele- embedded in XML schemata in complex systems. This paper ments in a XML schema, which therefore is a useful starting proposes a mapping method for identifying relevant semantic point for semantic analysis. In such systems, XML schemata data in XML schemata, automatically structuring and repre- have been shown to be a valuable resource of semantic data senting it in the form of a draft ontology. Concepts, concept [2]. Command and Control systems in the military context hierarchy and domain relationships from XML schema are support various functions, including commanding of forces mapped to relevant parts of an OWL ontology. -
Semantic Role Labeling Tutorial NAACL, June 9, 2013
Semantic Role Labeling Tutorial NAACL, June 9, 2013 Part 1: Martha Palmer, University of Colorado Part 2: Shumin Wu, University of Colorado Part 3: Ivan Titov, Universität des Saarlandes 1 Outline } Part 1 Linguistic Background, Resources, Annotation Martha Palmer, University of Colorado } Part 2 Supervised Semantic Role Labeling and Leveraging Parallel PropBanks Shumin Wu, University of Colorado } Part 3 Semi- , unsupervised and cross-lingual approaches Ivan Titov, Universität des Saarlandes, Universteit van Amsterdam 2 Motivation: From Sentences to Propositions Who did what to whom, when, where and how? Powell met Zhu Rongji battle wrestle join debate Powell and Zhu Rongji met consult Powell met with Zhu Rongji Proposition: meet(Powell, Zhu Rongji) Powell and Zhu Rongji had a meeting meet(Somebody1, Somebody2) . When Powell met Zhu Rongji on Thursday they discussed the return of the spy plane. meet(Powell, Zhu) discuss([Powell, Zhu], return(X, plane)) Capturing semantic roles SUBJ } Dan broke [ the laser pointer.] SUBJ } [ The windows] were broken by the hurricane. SUBJ } [ The vase] broke into pieces when it toppled over. PropBank - A TreeBanked Sentence (S (NP-SBJ Analysts) S (VP have (VP been VP (VP expecting (NP (NP a GM-Jaguar pact) have VP (SBAR (WHNP-1 that) (S (NP-SBJ *T*-1) NP-SBJ been VP (VP would Analysts (VP give expectingNP (NP the U.S. car maker) SBAR (NP (NP an eventual (ADJP 30 %) stake) NP S (PP-LOC in (NP the British company)))))))))))) a GM-Jaguar WHNP-1 VP pact that NP-SBJ VP *T*-1 would NP give Analysts have been expecting a GM-Jaguar NP PP-LOC pact that would give the U.S. -
Natural Language Processing
Chowdhury, G. (2003) Natural language processing. Annual Review of Information Science and Technology, 37. pp. 51-89. ISSN 0066-4200 http://eprints.cdlr.strath.ac.uk/2611/ This is an author-produced version of a paper published in The Annual Review of Information Science and Technology ISSN 0066-4200 . This version has been peer-reviewed, but does not include the final publisher proof corrections, published layout, or pagination. Strathprints is designed to allow users to access the research output of the University of Strathclyde. Copyright © and Moral Rights for the papers on this site are retained by the individual authors and/or other copyright owners. Users may download and/or print one copy of any article(s) in Strathprints to facilitate their private study or for non-commercial research. You may not engage in further distribution of the material or use it for any profitmaking activities or any commercial gain. You may freely distribute the url (http://eprints.cdlr.strath.ac.uk) of the Strathprints website. Any correspondence concerning this service should be sent to The Strathprints Administrator: [email protected] Natural Language Processing Gobinda G. Chowdhury Dept. of Computer and Information Sciences University of Strathclyde, Glasgow G1 1XH, UK e-mail: [email protected] Introduction Natural Language Processing (NLP) is an area of research and application that explores how computers can be used to understand and manipulate natural language text or speech to do useful things. NLP researchers aim to gather knowledge on how human beings understand and use language so that appropriate tools and techniques can be developed to make computer systems understand and manipulate natural languages to perform the desired tasks.