DOCSLIB.ORG

A Matter of Framing: the Impact of Linguistic Formalism on Probing Results

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Matter of Framing: the Impact of Linguistic Formalism on Probing Results A Matter of Framing: The Impact of Linguistic Formalism on Probing Results Ilia Kuznetsov and Iryna Gurevych Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science, Technische Universitat¨ Darmstadt http://www.ukp.tu-darmstadt.de/ Abstract L=0 L=8 L=11 We Deep pre-trained contextualized encoders like asked BERT (Devlin et al., 2019) demonstrate re- John to markable performance on a range of down- leave stream tasks. A recent line of research in prob- . ing investigates the linguistic knowledge im- plicitly learned by these models during pre- Figure 1: Intra-sentence similarity by layer L of the training. While most work in probing oper- multilingual BERT-base. Functional tokens are similar ates on the task level, linguistic tasks are rarely in L = 0, syntactic groups emerge at higher levels. uniform and can be represented in a variety of formalisms. Any linguistics-based probing study thereby inevitably commits to the for- linguistic knowledge (Reif et al., 2019). Indeed, malism used to annotate the underlying data. even an informal inspection of layer-wise intra- Can the choice of formalism affect probing re- sults? To investigate, we conduct an in-depth sentence similarities (Fig.1) suggests that these cross-formalism layer probing study in role se- models capture elements of linguistic structure, and mantics. We find linguistically meaningful dif- those differ depending on the layer of the model. ferences in the encoding of semantic role- and A grounded investigation of these regularities al- proto-role information by BERT depending on lows to interpret the model’s behaviour, design the formalism and demonstrate that layer prob- better pre-trained encoders and inform the down- ing can detect subtle differences between the stream model development. Such investigation is implementations of the same linguistic formal- the main subject of probing, and recent studies con- ism. Our results suggest that linguistic formal- ism is an important dimension in probing stud- firm that BERT implicitly captures many aspects ies and should be investigated along with the of language use, lexical semantics and grammar commonly used cross-task and cross-lingual (Rogers et al., 2020). experimental settings. Most probing studies use linguistics as a theoret- ical scaffolding and operate on a task level. How- 1 Introduction ever, there often exist multiple ways to represent The emergence of deep pre-trained contextualized the same linguistic phenomenon: for example, En- encoders has had a major impact on the field of nat- glish dependency syntax can be encoded using a ural language processing. Boosted by the availabil- variety of formalisms, incl. Universal (Schuster ity of general-purpose frameworks like AllenNLP and Manning, 2016), Stanford (de Marneffe and (Gardner et al., 2018) and Transformers (Wolf Manning, 2008) and CoNLL-2009 dependencies et al., 2019), pre-trained models like ELMO (Pe- (Hajicˇ et al., 2009), all using different label sets and ters et al., 2018) and BERT (Devlin et al., 2019) syntactic head attachment rules. Any probing study have caused a shift towards simple architectures inevitably commits to the specific theoretical frame- where a strong pre-trained encoder is paired with a work used to produce the underlying data. The dif- shallow downstream model, often outperforming ferences between linguistic formalisms, however, the intricate task-specific architectures of the past. can be substantial. The versatility of pre-trained representations im- Can these differences affect the probing results? plies that they encode some aspects of general This question is intriguing for several reasons. Lin- 171 Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 171–182, November 16–20, 2020. c 2020 Association for Computational Linguistics guistic formalisms are well-documented, and if the formalism probing study on role semantics, thereby choice of formalism indeed has an effect on prob- contributing to the line of research on how and ing, cross-formalism comparison will yield new whether pre-trained BERT encodes higher-level se- insights into the linguistic knowledge obtained by mantic phenomena. contextualized encoders during pre-training. If, alternatively, the probing results remain stable de- Contributions. This work studies the effect of spite substantial differences between formalisms, the linguistic formalism on probing results. We con- this prompts a further scrutiny of what the pre- duct cross-formalism experiments on PropBank, trained encoders in fact encode. Finally, on the re- VerbNet and FrameNet role prediction in English verse side, cross-formalism probing might be used and German, and show that the formalism can af- as a tool to empirically compare the formalisms fect probing results in a linguistically meaningful and their language-specific implementations. To way; in addition, we demonstrate that layer probing the best of our knowledge we are the first to explic- can detect subtle differences between implementa- itly address the influence of formalism on probing. tions of the same formalism in different languages. On the technical side, we advance the recently intro- Ideally, the task chosen for a cross-formalism duced edge and layer probing framework (Tenney study should be encoded in multiple formalisms et al., 2019b); in particular, we introduce anchor using the same textual data to rule out the influ- tasks - an analytical tool inspired by feature-based ence of the domain and text type. While many lin- systems that allows deeper qualitative insights into guistic corpora contain several layers of linguistic the pre-trained models’ behaviour. Finally, ad- information, having the same textual data anno- vancing the current knowledge about the encod- tated with multiple formalisms for the same task ing of predicate semantics in BERT, we perform a is rare. We focus on role semantics – a family fine-grained semantic proto-role probing study and of shallow semantic formalisms at the interface demonstrate that semantic proto-role properties between syntax and propositional semantics that can be extracted from pre-trained BERT, con- assign roles to the participants of natural language trary to the existing reports. Our results suggest utterances, determining who did what to whom, that along with task and language, linguistic for- where, when etc. Decades of research in theo- malism is an important dimension to be accounted retical linguistics have produced a range of role- for in probing research. semantic frameworks that have been operational- ized in NLP: syntax-driven PropBank (Palmer et al., 2 Related Work 2005), coarse-grained VerbNet (Kipper-Schuler, 2005), fine-grained FrameNet (Baker et al., 1998), 2.1 BERT as Encoder and, recently, decompositional Semantic Proto- BERT is a Transformer (Vaswani et al., 2017) en- Roles (SPR) (Reisinger et al., 2015; White et al., coder pre-trained by jointly optimizing two unsu- 2016). The SemLink project (Bonial et al., 2013) pervised objectives: masked language model and offers parallel annotation for PropBank, VerbNet next sentence prediction. It uses WordPiece (WP, and FrameNet for English. This allows us to iso- Wu et al.(2016)) subword tokens along with po- late the object of our study: apart from the role- sitional embeddings as input, and gradually con- semantic labels, the underlying data and conditions structs sentence representations by applying token- for the three formalisms are identical. SR3DE level self-attention pooling over a stack of layers L. (Mujdricza-Maydt´ et al., 2016) provides compati- The result of BERT encoding is a layer-wise repre- ble annotation in three formalisms for German, en- sentation of the input wordpiece tokens with higher abling cross-lingual validation of our results. Com- layers representing higher-level abstractions over bined, these factors make role semantics an ideal the input sequence. Thanks to the joint pre-training target for our cross-formalism probing study. objective, BERT can encode words and sentences A solid body of evidence suggests that encoders in a unified fashion: the encoding of a sentence or like BERT capture syntactic and lexical-semantic a sentence pair is stored in a special token [CLS]. properties, but only few studies have considered To facilitate multilingual experiments, we use probing for predicate-level semantics (Tenney et al., the multilingual BERT-base (mBERT) published 2019b; Kovaleva et al., 2019). To the best of by Devlin et al.(2019). Although several recent our knowledge we are the first to conduct a cross- encoders have outperformed BERT on benchmarks 172 (Liu et al., 2019; Lan et al., 2019; Raffel et al., Our probing methodology builds upon the edge 2019), we use the original BERT architecture, since and layer probing framework. The encoding pro- it allows us to inherit the probing methodology and duced by a frozen BERT model can be seen as a to build upon the related findings. layer-wise snapshot that reflects how the model has constructed the high-level abstractions. Tenney 2.2 Probing et al.(2019b) introduce the edge probing task de- sign: a simple classifier is tasked with predicting Due to space limitations we omit high-level dis- a linguistic property given a pair of spans encoded cussions on benchmarking (Wang et al., 2018) and using a frozen pre-trained model. Tenney et al. sentence-level probing (Conneau et al., 2018a), and (2019a) use edge probing to analyse the layer uti- focus on the recent findings related to the represen- lization of a pre-trained BERT model via scalar tation of linguistic structure in BERT. Surface-level mixing weights (Peters et al., 2018) learned during information generally tends to be represented in the training. We revisit this framework in Section3. lower layers of deep encoders, while higher layers store hierarchical and semantic information (Be- 2.3 Role Semantics linkov et al., 2017; Lin et al., 2019). Tenney et al. (2019a) show that the abstraction strategy applied We now turn to the object of our investigation: role by the English pre-trained BERT encoder follows semantics.
Load more

Recommended publications
  • Synchronous Dependency Insertion Grammars a Grammar Formalism for Syntax Based Statistical MT
    Synchronous Dependency Insertion Grammars A Grammar Formalism for Syntax Based Statistical MT Yuan Ding and Martha Palmer Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104, USA {yding, mpalmer}@linc.cis.upenn.edu Abstract In the early 1990s, (Brown et. al. 1993) intro- duced the idea of statistical machine translation, This paper introduces a grammar formalism where the word to word translation probabilities and specifically designed for syntax-based sta- sentence reordering probabilities are estimated from tistical machine translation. The synchro- a large set of parallel sentence pairs. By having the nous grammar formalism we propose in advantage of leveraging large parallel corpora, the this paper takes into consideration the per- statistical MT approach outperforms the traditional vasive structure divergence between lan- transfer based approaches in tasks for which ade- guages, which many other synchronous quate parallel corpora is available (Och, 2003). grammars are unable to model. A Depend- However, a major criticism of this approach is that it ency Insertion Grammars (DIG) is a gen- is void of any internal representation for syntax or erative grammar formalism that captures semantics. word order phenomena within the depend- In recent years, hybrid approaches, which aim at ency representation. Synchronous Depend- applying statistical learning to structured data, began ency Insertion Grammars (SDIG) is the to emerge. Syntax based statistical MT approaches synchronous version of DIG which aims at began with (Wu 1997), who introduced a polyno- capturing structural divergences across the mial-time solution for the alignment problem based languages. While both DIG and SDIG have on synchronous binary trees.
    [Show full text]
  • Functional Unification Grammar: a Formalism for Machine Translation
    FUNCTIONAL UNIFICATION GRAMMAR: A FORMALISM FOR MACHINE TRANSLATION Martin Kay Xerox Palo Alto Research Center 3333 Coyote Hill Road Palo Alto California 94304 and CSLI, Stanford Abstract language--morphological, syntactic, semantic, or whatever--could be stated. A formalism powerful enough to accommodate the Functional Unification Grammar provides an opportunity various different kinds of linguistic phenomena with equal facility to encompass within one formalism and computational system might be unappealing to theoretical linguists because powerful the parts of machine translation systems that have usually been formal systems do not make powerful claims. But the engineering treated separately, natably analysis, transfer, and synthesis. advantages are clear to see. A single formalism would straightfor- Many of the advantages of this formalism come from the fact wardly reduce the number of interpreters to two, one for analysis that it is monotonic allowing data structures to grow differently and one for synthesis. Furthermore, the explanatory value of a as different nondeterministic alternatives in a computation are theory clearly rests on a great deal more than the restriciveness of pursued, but never to be modified in any way. A striking feature its formal base. In particular, the possiblity of encompassing what of this system is that it is fundamental reversible, allowing a to had hitherto been thought to require altogether different kinds of translate as b only if b could translate as a. treatment within a single framework could be theoretically inter- esting. I Overview Another clear improvement on the classical design would A. Machine Translation "result from merging 'the two interpreters associated with a for- malism.
    [Show full text]
  • Wittgenstein and Musical Formalism: a Case Revisited
    Ápeiron. Estudios de filosofía — Monográfico «Wittgenstein. Música y arquitectura» Wittgenstein and Musical Formalism: A Case Revisited Wittgenstein y el formalismo musical: Un caso reconsiderado Hanne Appelqvist University of Helsinki [email protected] Abstract: This article defends a formalist interpretation of Wittgenstein’s later thought on music by com- paring it with Eduard Hanslick’s musical formalism. In doing so, it returns to a disagreement I have had with Bela Szabados who, in his book Wittgenstein as a Philosophical Tone-Poet, claims that the attribution of formalism obscures the role that music played in the development of Wittgenstein’s thought. The paper scrutinizes the four arguments Szabados presents to defend his claim, pertaining to alleged differences be- tween Wittgenstein and Hanslick on their accounts of theory, beauty, rules, and the broader significance of music. I will argue that in each case the similarities between Wittgenstein’s and Hanslick’s respective views outshine possible differences. Ultimately, I will argue that instead of rendering music a marginal phenom- enon suited for mere entertainment, formalism –as presented by Hanslick and Wittgenstein, whom I read as influenced by Kant’s aesthetics– underscores music’s ability to show fundamental features of reality and our relation to it. Music does this precisely as a sensuous yet structured medium that is irreducible to any conceptually determined domain. Keywords: Wittgenstein, Hanslick, Kant, formalism, music. Resumen: Este artículo defiende una interpretación formalista del pensamiento posterior de Wittgenstein so- bre la música comparándolo con el formalismo musical de Eduard Hanslick. Con ese fin, reconsidera un des- acuerdo que he tenido con Bela Szabados.
    [Show full text]
  • Ten Choices for Lexical Semantics
    1 Ten Choices for Lexical Semantics Sergei Nirenburg and Victor Raskin1 Computing Research Laboratory New Mexico State University Las Cruces, NM 88003 {sergei,raskin}@crl.nmsu.edu Abstract. The modern computational lexical semantics reached a point in its de- velopment when it has become necessary to define the premises and goals of each of its several current trends. This paper proposes ten choices in terms of which these premises and goals can be discussed. It argues that the central questions in- clude the use of lexical rules for generating word senses; the role of syntax, prag- matics, and formal semantics in the specification of lexical meaning; the use of a world model, or ontology, as the organizing principle for lexical-semantic descrip- tions; the use of rules with limited scope; the relation between static and dynamic resources; the commitment to descriptive coverage; the trade-off between general- ization and idiosyncracy; and, finally, the adherence to the “supply-side” (method- oriented) or “demand-side” (task-oriented) ideology of research. The discussion is inspired by, but not limited to, the comparison between the generative lexicon ap- proach and the ontological semantic approach to lexical semantics. It is fair to say that lexical semantics, the study of word meaning and of its representation in the lexicon, experienced a powerful resurgence within the last decade.2 Traditionally, linguistic se- mantics has concerned itself with two areas, word meaning and sentence meaning. After the ascen- dancy of logic in the 1930s and especially after “the Chomskian revolution” of the late 1950s-early 1960s, most semanticists, including those working in the transformational and post-transforma- tional paradigm, focused almost exclusively on sentence meaning.
    [Show full text]
  • The Computational Lexical Semantics of Syntagmatic Expressions
    The Computational Lexical Semantics of Syntagmatic Relations Evelyne Viegas, Stephen Beale and Sergei Nirenburg New Mexico State University Computing Research Lab, Las Cruces, NM 88003, USA viegas, sb, sergei©crl, nmsu. edu Abstract inheritance hierarchy of Lexical Semantic Functions In this paper, we address the issue of syntagmatic (LSFs). expressions from a computational lexical semantic perspective. From a representational viewpoint, we 2 Approaches to Syntagmatic argue for a hybrid approach combining linguistic and Relations conceptual paradigms, in order to account for the Syntagmatic relations, also known as collocations, continuum we find in natural languages from free are used differently by lexicographers, linguists and combining words to frozen expressions. In particu- statisticians denoting almost similar but not identi- lar, we focus on the place of lexical and semantic cal classes of expressions. restricted co-occurrences. From a processing view- The traditional approach to collocations has been point, we show how to generate/analyze syntag- lexicographic. Here dictionaries provide infor- matic expressions by using an efficient constraint- mation about what is unpredictable or idiosyn- based processor, well fitted for a knowledge-driven cratic. Benson (1989) synthesizes Hausmann's stud- approach. ies on collocations, calling expressions such as com- 1 Introduction mit murder, compile a dictionary, inflict a wound, etc. "fixed combinations, recurrent combinations" You can take advantage o] the chambermaid 1 is not a or "collocations". In Hausmann's terms (1979) a collocation one would like to generate in the context collocation is composed of two elements, a base ("Ba- of a hotel to mean "use the services of." This is why sis") and a collocate ("Kollokator"); the base is se- collocations should constitute an important part in mantically autonomous whereas the collocate cannot the design of Machine Translation or Multilingual be semantically interpreted in isolation.
    [Show full text]
  • The Semantics of Grammar Formalisms Seen As Computer Languages
    The semantics of grammar formalisms seen as computer languages The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Fernando C. N. Pereira and Stuart M. Shieber. The semantics of grammar formalisms seen as computer languages. In Proceedings of the Tenth International Conference on Computational Linguistics, pages 123-129, Stanford University, Stanford, California, July 2-6 1984. Published Version http://dx.doi.org/10.3115/980491.980518 Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:2309658 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA The Semantics of Grammar Formalisms Seen as Computer Languages Fernando C. N. Pereira and Stuart M. Shieber Artificial Intelligence Center SRI International and Center for the Study of Language and Information Stanford University Abstract a declarative semantics z extended in the natural way from the declarative semantics of context-free grammars. The design, implementation, and use of grammar for- The last point deserves amplification. Context-free ma]isms for natural language have constituted a major grammars possess an obvious declarative semantics in branch of coml)utational linguistics throughout its devel- which nonterminals represent sets of strings and rules rep- opment. By viewing grammar formalisms as just a spe- resent n-ary relations over strings. This is brought out by cial ease of computer languages, we can take advantage of the reinterpretation familiar from formal language theory the machinery of denotational semantics to provide a pre- of context-free grammars as polynomials over concatena- cise specification of their meaning.
    [Show full text]
  • Pragmatic Formalism, Separation of Powers, and the Need to Revisit the Nondelegation Doctrine
    Loyola University Chicago Law Journal Volume 51 Issue 2 Winter 2019 Article 4 2019 Pragmatic Formalism, Separation of Powers, and the Need to Revisit the Nondelegation Doctrine Martin H. Redish Follow this and additional works at: https://lawecommons.luc.edu/luclj Part of the Law Commons Recommended Citation Martin H. Redish, Pragmatic Formalism, Separation of Powers, and the Need to Revisit the Nondelegation Doctrine, 51 Loy. U. Chi. L. J. 363 (2020). Available at: https://lawecommons.luc.edu/luclj/vol51/iss2/4 This Article is brought to you for free and open access by LAW eCommons. It has been accepted for inclusion in Loyola University Chicago Law Journal by an authorized editor of LAW eCommons. For more information, please contact [email protected]. Pragmatic Formalism, Separation of Powers, and the Need to Revisit the Nondelegation Doctrine Martin H. Redish* In a time where the executive branch continues to grow in size and strength, reviving the nondelegation doctrine has become more important than ever. Judicial enforcement of this abandoned rule that Congress cannot delegate its legislative power serves three vital functions. It preserves the separation of powers, prevents tyranny, and promotes democratic accountability. But even if we acknowledge that enforcement of the doctrine is necessary to preserve the American form of government, a more difficult question is how the doctrine ought to be enforced. This Article rejects the formulations of the nondelegation doctrine proposed by both functionalists and formalists and proposes “pragmatic formalism” as a solution that carves a path between the two existing theories. After explaining the theory and applying it to the facts of the Supreme Court’s major nondelegation cases, the Article also proposes a solution to one of the most difficult challenges facing the nondelegation doctrine: delegation during times of national emergency.
    [Show full text]
  • Russian Formalism: a Metapoetics
    Russian Formalism Russian For1nalis1n A Metapoetics by PETER STEINER Cornell University Press I Ithaca I London Copyright © 1984 by Peter Steiner All rights reserved. Except for brief quotations in a review, this book, or parts thereof, must not be reproduced in any form without permission in writing from the publisher. For information, address Cornell University Press, Sage House, 512 East State Street, Ithaca, New York 14850, or visit our website at www.cornellpress.cornell.edu. First published 1984 by Cornell University Press. “Too Much Monkey Business,” by Chuck Berry, copyright © 1956, Arc Music Corp., 110 East 59th Street, New York, N.Y. 10022. International copyright secured. Used by permission. Library of Congress Cataloging-in-Publication Data Steiner, P. (Peter), 1946– Russian formalism. Based on the author’s thesis. Includes index. 1. Formalism (Literary analysis)—Soviet Union. I. Title. PN98.F6S73 1984 801'.95 84-7708 ISBN 978-0-8014-1710-8 (pbk. : alk. paper) The text of this book is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License: https://creativecommons.org/licenses/by-nc-nd/4.0/ Blond hair, good look'n', wants me to marry, get a home, settle down, write a book- ahhhhhhhhhhhh! Too much monkey business, too much monkey business. -CHUCK BERRY Contents Preface 9 1 Who Is Formalism, What Is She? 15 � The Three Metaphors 44 The Machine 44 The Organism 68 The System 99 3 A Synecdoche 1]8 Zaum' 140 Verse 172 Expression 199 4 The Developmental Significance of Russian Formalism 24 2 Index 2 71 7 Preface This book grew out of my earlier comparative study of Rus­ sian Formalism and Prague Structuralism.
    [Show full text]
  • The Algebra of Language Introduction to Syntax
    The$Algebra$of$Language$an$Introduction$to$ Syntax$ Adam$Szczegielniak Rutgers$University [email protected] Ver.$2 Copyright:$Adam$Szczegielniak TOC: Introduction/…………………………………..2 §1Sentence/structure………………………6 §2//Graphs………………………………………14 §3/Word/categories………………………….23 §4/XABar…………………………………………..47 §5/XABar,/Part/2/Functional/Heads……..78 §6CAcommand/and/Binding/……………..100 §7/Movement/Intro…………………………..110 §7.1/Head/Movement………………………..121 §8/DP/movement/……………………………...136 §9/WhAmovement/……………………………..162 1 /Introduction This/is/a/script/designed/to/assist/students/who/are/taking/an/introduction/to/syntax/ class/as/part/of/the/theoretical/linguistics/curriculum./It/is/not/really/designed/as/a/stand/ alone/text/and/is/Best/used/in/conduction/with/class/lectures./The/framework/adopted/in/this/ textBook/is/Generative/Grammar./There/is/no/historical/preview/of/the/Wield./You/will/not/Wind/ in/here/discussion/of/phrase/structure/rules,/which/were/part/of/early/research/in/the/Wield,/ But/you/will/also/not/Wind/more/recent/versions/of/Minimalism./This/is/an/introduction/text/ and/as/such/its/aim/is/to/assist/in/training/you/to/think/like/an/empirical/scientist/and/a/ linguist./For/some,/this/might/Be/a/surprise/that/linguistics/is/aiming/for/the/empiricalA/ scientiWic/method,/as/opposed/to/some/other/more/neBulous/approach/that/is,/often/ mistakenly,/associated/with/the/humanities./This/stems/from/the/very/nature/of/the/ Chomsky/revolution/of/the/1950’s/that/gave/rise/to/Generative/Grammar./Linguistics/is/the/ study/of/what/a/speaker/knows/when/she/he/is/said/to/speak/a/given/language./It/is/a/way/to/
    [Show full text]
  • Form and Formalism in Linguistics
    Form and formalism in linguistics Edited by James McElvenny language History and Philosophy of the Language science press Sciences 1 History and Philosophy of the Language Sciences Editor: James McElvenny In this series: 1. McElvenny, James (ed.). Form and formalism in linguistics. Form and formalism in linguistics Edited by James McElvenny language science press McElvenny, James (ed.). 2019. Form and formalism in linguistics (History and Philosophy of the Language Sciences 1). Berlin: Language Science Press. This title can be downloaded at: http://langsci-press.org/catalog/book/214 © 2019, the authors Published under the Creative Commons Attribution 4.0 Licence (CC BY 4.0): http://creativecommons.org/licenses/by/4.0/ ISBN: 978-3-96110-182-5 (Digital) 978-3-96110-183-2 (Hardcover) DOI:10.5281/zenodo.2654375 Source code available from www.github.com/langsci/214 Collaborative reading: paperhive.org/documents/remote?type=langsci&id=214 Cover and concept of design: Ulrike Harbort Typesetting: James McElvenny Proofreading: Agnes Kim, Andreas Hölzl, Brett Reynolds, Daniela Hanna-Kolbe, Els Elffers, Eran Asoulin, George Walkden, Ivica Jeđud, Jeroen van de Weijer, Judith Kaplan, Katja Politt, Lachlan Mackenzie, Laura Melissa Arnold, Nick Riemer, Tom Bossuyt, Winfried Lechner Fonts: Linux Libertine, Libertinus Math, Arimo, DejaVu Sans Mono Typesetting software:Ǝ X LATEX Language Science Press Unter den Linden 6 10099 Berlin, Germany langsci-press.org Storage and cataloguing done by FU Berlin Contents Preface James McElvenny iii 1 Visual formalisms in comparative-historical linguistics Judith Kaplan 1 2 Alternating sounds and the formal franchise in phonology James McElvenny 35 3 On Sapir’s notion of form/pattern and its aesthetic background Jean-Michel Fortis 59 4 Linguistics as a “special science”: A comparison of Sapir and Fodor Els Elffers 89 5 The impact of Russian formalism on linguistic structuralism Bart Karstens 115 6 The resistant embrace of formalism in the work of Émile Benveniste and Aurélien Sauvageot John E.
    [Show full text]
  • A Formalism and Environment for the Development of a Large Grammar of English
    A Formalism and Environment for the Development of a Large Grammar of English Ted Briscoe, Claire Grover Bran Boguraev, John Carroll Department of Linguistics, University of Lancaster, Computer Laboratory, University of Cambridge, Bailrigg, Lancaster LAI 4YT, England Com Exchange Street, Cambridge CB2 3QG, England ABSTRACT II THE METAGRAMMATICAL NOTATION Natural language grammars with large coverage are typically The metagrammatical notation we have designed is based on the result of many person-years of effort, working with clumsy GPSG, but has been modified and extended to be more flexible formalisms and sub-optimal software support for grammar and expressive and is interpreted somewhat differently. The first development. This paper describes our approach to the task of motivation for these changes is to define a notation which writing a substantial grammar, as part of a collaboration to provides a specialised programming language for specifying produce a general purpose morphological and syntactic analyser grammatical theories and grammars for particular languages, for English. The grammatical formalism we have developed for rather than defining a specific theory directly. This approach both the task is a metagrammatical notation which is a more ensures that developments in syntactic theory will be less likely expressive and computationally tractable variant of Generalized to render the grammar development environment obsolete and Phrase Structure Grammar. We have also implemented a software also provides the grammarian with an expressively flexible and system which provides a highly integrated and very powerful set powerful notation for grammar writing, rather than a formalism of tools for developing and managing a large grammar based on embodying a restrictive theory.
    [Show full text]
  • Knowledge Representation Formalism for Building Semantic Web Ontologies
    Knowledge Representation Formalism For Building Semantic Web Ontologies by Basak Taylan A survey submitted to the Graduate Faculty in Computer Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy, The City University of New York. 2017 Contents 1 INTRODUCTION 6 2 SEMANTIC WEB 11 3 KNOWLEDGE REPRESENTATION 24 3.1 KNOWLEDGE . 24 3.2 KNOWLEDGE REPRESENTATION FORMALISM . 27 3.2.1 Roles Of Knowledge Representation . 27 3.3 KNOWLEDGE REPRESENTATION METHODS . 33 3.3.1 NETWORKED REPRESENTATION . 33 3.3.1.1 SEMANTIC NETWORKS . 33 3.3.1.2 CONCEPTUAL GRAPHS . 39 3.3.2 STRUCTURAL REPRESENTATION . 42 3.3.2.1 FRAMES . 42 3.3.2.2 KL-ONE KNOWLEDGE REPRESENTATION SYSTEMS . 49 3.3.3 LOGIC-BASED REPRESENTATION . 53 3.3.3.1 PROPOSITIONAL LOGIC(PL) . 54 3.3.3.2 FIRST-ORDER LOGIC(FOL) . 58 2 CONTENTS 3 3.3.3.3 DESCRIPTION LOGICS . 64 3.3.3.4 FRAME LOGIC(F-LOGIC) . 76 4 APPLICATIONS 86 4.1 The Open Mind Common Sense Project(OMCS) . 86 4.2 ConceptNet 3: a Flexible, Multilingual Semantic Network for Common Sense Knowledge . 89 4.3 WordNet . 94 4.4 FrameNet . 99 4.5 VerbNet . 102 4.6 The Brandeis Semantic Ontology (BSO) . 107 5 CONCLUSION 109 Abstract The initial idea of Tim Berners Lee when designing The Wold Wide Web was creating a commonly accessible area in the network for information sharing by means of hyperlinks without concerning about platform dependency in late 1980s[28]. Since then, the Web has been growing dramatically and it has became a major information retrieval means.
    [Show full text]