DOCSLIB.ORG

Generating Lexical Representations of Frames Using Lexical Substitution

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Generating Lexical Representations of Frames Using Lexical Substitution Generating Lexical Representations of Frames using Lexical Substitution Saba Anwar Artem Shelmanov Universitat¨ Hamburg Skolkovo Institute of Science and Technology Germany Russia [email protected] [email protected] Alexander Panchenko Chris Biemann Skolkovo Institute of Science and Technology Universitat¨ Hamburg Russia Germany [email protected] [email protected] Abstract Seed sentence: I hope PattiHelper can helpAssistance youBenefited party soonTime . Semantic frames are formal linguistic struc- Substitutes for Assistance: assist, aid tures describing situations/actions/events, e.g. Substitutes for Helper: she, I, he, you, we, someone, Commercial transfer of goods. Each frame they, it, lori, hannah, paul, sarah, melanie, pam, riley Substitutes for Benefited party: me, him, folk, her, provides a set of roles corresponding to the sit- everyone, people uation participants, e.g. Buyer and Goods, and Substitutes for Time: tomorrow, now, shortly, sooner, lexical units (LUs) – words and phrases that tonight, today, later can evoke this particular frame in texts, e.g. Sell. The scarcity of annotated resources hin- Table 1: An example of the induced lexical represen- ders wider adoption of frame semantics across tation (roles and LUs) of the Assistance FrameNet languages and domains. We investigate a sim- frame using lexical substitutes from a single seed sen- ple yet effective method, lexical substitution tence. with word representation models, to automat- ically expand a small set of frame-annotated annotated resources. Some publicly available re- sentences with new words for their respective sources are FrameNet (Baker et al., 1998) and roles and LUs. We evaluate the expansion PropBank (Palmer et al., 2005), yet for many lan- quality using FrameNet. Contextualized mod- guages and domains, specialized resources do not els demonstrate overall superior performance exist. Besides, due to the inherent vagueness compared to the non-contextualized ones on of frame definitions, the annotation task is chal- roles. However, the latter show comparable performance on the task of LU expansion. lenging and requires semanticists or very complex crowd-sourcing setups (Fossati et al., 2013). 1 Introduction We suggest a different perspective on the prob- lem: expanding the FrameNet resource automati- The goal of lexical substitution (McCarthy and cally by using lexical substitution. Given a small Navigli, 2009) is to replace a given target word set of seed sentences with their frame annotations, in its context with meaning-preserving alterna- we can expand it by substituting the targets (words tives. In this paper, we show how lexical sub- corresponding to lexical units of the respective stitution can be used for semantic frame expan- frame) and arguments (words corresponding to sion. A semantic frame is a linguistic structure roles of the respective frame) of those sentences used to describe the formal meaning of a situa- and aggregating possible substitutions into an in- tion or event (Fillmore, 1982). Semantic frames duced frame-semantic resource. Table1 shows have witnessed a wide range of applications; such one such induced example. For this purpose, as question answering (Shen and Lapata, 2007; we have experimented with state-of-the-art non- Berant and Liang, 2014; Khashabi et al., 2018), contextualized (static) word representation mod- machine translation (Gao and Vogel, 2011; Zhai els including neural word embeddings, i.e. fast- et al., 2013), and semantic role labelling (Do et al., Text (Bojanowski et al., 2017), GloVe (Pennington 2017; Swayamdipta et al., 2018). The impact, et al., 2014), and word2vec (Mikolov et al., 2013); however, is limited by the scarce availability of and distributional thesaurus, i.e. JoBimText (Bie- 95 Proceedings of the Probability and Meaning Conference (PaM 2020), pages 95–103 October 14-15 2020 c 2020 Association for Computational Linguistics mann and Riedl, 2013); and compared their results WordNet-based methods to automatically induce with contextualized word representations of the new LUs and reported their results on FrameNet. state-of-the-art BERT model (Devlin et al., 2019), Our method is inspired by the recent work of which has set a new benchmark performance on Amrami and Goldberg(2018). They suggest to many downstream NLP applications. To complete predict the substitutes vectors for target words us- the comparison, we also include the lexical substi- ing pre-trained ELMo (Peters et al., 2018) and dy- tution model of Melamud et al.(2015), which uses namic symmetric patterns, then induced the word dependency-based word and context embeddings senses using clustering. Arefyev et al.(2019) takes and produces context-sensitive lexical substitutes. the idea of substitute vectors from (Amrami and To generate substitutes, we decompose the Goldberg, 2018) for the SemEval 2019 (Qasem- problem into two sub-tasks: Lexical unit expan- iZadeh et al., 2019) frame induction task and re- sion: Given a sentence and its target word, the task places ELMo with BERT (Devlin et al., 2019) for is to generate frame preserving substitutes for this improved performance. Zhou et al.(2019) show word. Frame role expansion: Given a sentence the utility of BERT for the lexical substitution and an argument, the task is to generate meaning- task. Lexical substitution has been used for a preserving substitutes for this argument. range of NLP tasks such as paraphrasing or text Contributions of our work are (i) a method for simplification, but here, we are employing it, as inducing frame-semantic resources based on a few far as we are aware, for the first time to perform frame-annotated sentences using lexical substitu- expansion of frame-semantic resources. tion, and (ii) an evaluation of various distribu- tional semantic models and lexical substitution 3 Inducing Lexical Representations of methods on the ground truth from FrameNet. Frames via Lexical Substitution 2 Related Work We experimented with two groups of lexical sub- stitution methods. The first one use no con- Approaches to semantic frame parsing with re- text: non-contextualized neural word embed- spect to a pre-defined semantic frame resource, ding models, i.e. fastText (Bojanowski et al., such as FrameNet, have received much atten- 2017), GloVe (Pennington et al., 2014), and tion in the literature (Das et al., 2010; Oepen word2vec (Mikolov et al., 2013), as well as dis- et al., 2016; Yang and Mitchell, 2017; Peng et al., tributional thesaurus based models in the form of 2018), with SEMAFOR (Das et al., 2014) be- JoBimText (Biemann and Riedl, 2013). The sec- ing a most widely known system to extract com- ond group of methods does use the context: here, plete frame structure including target identifica- we tried contextualized word embedding model tion. Some works focus on identifying par- BERT (Devlin et al., 2019) and the lexical substi- tial structures such as frame identification (Hart- tution model of Melamud et al.(2015). mann et al., 2017; Hermann et al., 2014), role labelling with frame identification (Swayamdipta 3.1 Static Word Representations et al., 2017; Yang and Mitchell, 2017), and sim- These word representations models are inherently ple role labelling (Kshirsagar et al., 2015; Roth non-contextualized as they learn one representa- and Lapata, 2015; Swayamdipta et al., 2018), tion of a word regardless of its context. which is considered very similar to standard Prop- Bank (Palmer et al., 2005) style semantic role la- Neural Word Embeddings Neural word em- belling, albeit more challenging because of the beddings represent words as vectors of continu- high granularity of frame roles. These super- ous numbers, where words with similar meanings vised models rely on a dataset of frame-annotated are expected to have similar vectors. Thus, to pro- sentences such as FrameNet. FrameNet-like re- duce substitutes, we extracted the k nearest neigh- sources are available only for very few languages bors using a cosine similarity measure. We use and cover only a few domains. In this paper, we pre-trained embeddings by authors models: fast- venture into the inverse problem, the case where Text trained on the Common Crawl corpus, GloVe the number of annotations is insufficient, simi- trained on Common Crawl corpus with 840 billion lar to the idea of Pennacchiotti et al.(2008) who words, word2vec trained on Google News. All investigated the utility of semantic spaces and these models produce 300-dimension vectors. 96 Distributional Thesaurus (DT) In this ap- implementation by Wolf et al.(2019). We produce proach, word similarities are computed using com- a substitute word distribution without masking and plex linguistic features such as dependency rela- selected substitutes with top k probabilities. tions (Lin, 1998). The representations provided by DTs are sparser, but similarity scores based 4 Experimental Setup on them can be better. JoBimText (Biemann and Riedl, 2013) is a framework that offers many DTs 4.1 Datasets computed on a range of different corpora. Context We experimented with FrameNet (Baker et al., features for each word are ranked using the lexi- 1998) version 1:7. It contains around 170k sen- cographer’s mutual information (LMI) score and tences annotated with 1; 014 frames, 7; 878 types used to compute word similarity by feature over- of frame roles, and 10; 340 lexical units. Frame lap. We extract the k nearest neighbors for the tar- roles and LUs can consist of a single token or mul- get word. We use two JoBimText DTs: (i) DT tiple tokens. For this work, we have only con- built on Wikipedia with n-grams as contexts and sidered a single-token substitution. The datasets (ii) DT built on a 59G corpus (Wikipedia, Giga- for evaluation were derived automatically from word, ukWaC, and LCC corpora combined) using FrameNet. To create a gold standard for LU ex- dependency relations as context. pansion task, for each sentence containing an an- 3.2 Contextualized Models notated LU, we consider other LUs of the cor- responding semantic frame as ground truth sub- Static word representations fail to handle poly- stitutes.
Load more

Recommended publications
  • Abstraction and Generalisation in Semantic Role Labels: Propbank
    Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both? Paola Merlo Lonneke Van Der Plas Linguistics Department Linguistics Department University of Geneva University of Geneva 5 Rue de Candolle, 1204 Geneva 5 Rue de Candolle, 1204 Geneva Switzerland Switzerland [email protected] [email protected] Abstract The role of theories of semantic role lists is to obtain a set of semantic roles that can apply to Semantic role labels are the representa- any argument of any verb, to provide an unam- tion of the grammatically relevant aspects biguous identifier of the grammatical roles of the of a sentence meaning. Capturing the participants in the event described by the sentence nature and the number of semantic roles (Dowty, 1991). Starting from the first proposals in a sentence is therefore fundamental to (Gruber, 1965; Fillmore, 1968; Jackendoff, 1972), correctly describing the interface between several approaches have been put forth, ranging grammar and meaning. In this paper, we from a combination of very few roles to lists of compare two annotation schemes, Prop- very fine-grained specificity. (See Levin and Rap- Bank and VerbNet, in a task-independent, paport Hovav (2005) for an exhaustive review). general way, analysing how well they fare In NLP, several proposals have been put forth in in capturing the linguistic generalisations recent years and adopted in the annotation of large that are known to hold for semantic role samples of text (Baker et al., 1998; Palmer et al., labels, and consequently how well they 2005; Kipper, 2005; Loper et al., 2007). The an- grammaticalise aspects of meaning.
    [Show full text]
  • A Novel Large-Scale Verbal Semantic Resource and Its Application to Semantic Role Labeling
    VerbAtlas: a Novel Large-Scale Verbal Semantic Resource and Its Application to Semantic Role Labeling Andrea Di Fabio}~, Simone Conia}, Roberto Navigli} }Department of Computer Science ~Department of Literature and Modern Cultures Sapienza University of Rome, Italy {difabio,conia,navigli}@di.uniroma1.it Abstract The automatic identification and labeling of ar- gument structures is a task pioneered by Gildea We present VerbAtlas, a new, hand-crafted and Jurafsky(2002) called Semantic Role Label- lexical-semantic resource whose goal is to ing (SRL). SRL has become very popular thanks bring together all verbal synsets from Word- to its integration into other related NLP tasks Net into semantically-coherent frames. The such as machine translation (Liu and Gildea, frames define a common, prototypical argu- ment structure while at the same time pro- 2010), visual semantic role labeling (Silberer and viding new concept-specific information. In Pinkal, 2018) and information extraction (Bas- contrast to PropBank, which defines enumer- tianelli et al., 2013). ative semantic roles, VerbAtlas comes with In order to be performed, SRL requires the fol- an explicit, cross-frame set of semantic roles lowing core elements: 1) a verb inventory, and 2) linked to selectional preferences expressed in terms of WordNet synsets, and is the first a semantic role inventory. However, the current resource enriched with semantic information verb inventories used for this task, such as Prop- about implicit, shadow, and default arguments. Bank (Palmer et al., 2005) and FrameNet (Baker We demonstrate the effectiveness of VerbAtlas et al., 1998), are language-specific and lack high- in the task of dependency-based Semantic quality interoperability with existing knowledge Role Labeling and show how its integration bases.
    [Show full text]
  • Allied Irish Bank (GB) Comes Top Again in Comprehensive UK Banking Survey 27Th November 2000
    Allied Irish Bank (GB) comes top again in comprehensive UK banking survey 27th November 2000 Allied Irish Bank (GB) has today been named Best Business Bank for the fourth consecutive time in the Forum of Private Business’s (FPB) comprehensive survey into the strength of service offered by banks to private businesses. The FPB report, Private Businesses and Their Banks 2000, is a biennial survey of tens of thousands of British businesses and shows that Allied Irish Bank (GB) has maintained its No. 1 position over other major UK banks since 1994. Aidan McKeon, General Manager of Allied Irish Bank (GB) and Managing Director AIB Group (UK) p.l.c., commented: "While we are delighted to win this award for the fourth time, we are far from complacent. We continue to listen closely to our customers and to invest in the cornerstones of our business: recruiting, training and retaining quality people; building 'true’ business relationships; and ongoing commitment to maintaining short lines of decision making. At the same time, we are exploiting technology to make our service as customer- responsive and efficient as possible." Allied Irish Bank (GB), one of the forerunners in relationship banking, scores highest in the survey for knowledge and understanding. The bank also scored highly on efficiency, reliability and customer satisfaction. Mr. McKeon continued: "We recognise that business customers have particular needs and concerns and we are always striving to ensure that our customers receive a continually improved service. We shall look carefully at this survey and liaise with our customers to further strengthen our service." Stan Mendham, Chief Executive of the FPB commented: "The FPB congratulates Allied Irish Bank (GB) on being voted Best Business Bank in Britain for the fourth time.
    [Show full text]
  • Annual-Financial-Report-2009.Pdf
    Contents 4 Chairman’s statement 255 Statement of Directors’ responsibilities in relation to the Accounts 6 Group Chief Executive’s review 256 Independent auditor’s report 8 Corporate Social Responsibility 258 Additional information 12 Financial Review 276 Principal addresses - Business description 278 Index - Financial data - 5 year financial summary - Management report - Capital management - Critical accounting policies - Deposits and short term borrowings - Financial investments available for sale - Financial investments held to maturity - Contractual obligations - Off balance sheet arrangements 59 Risk Management - Risk Factors - Framework - Individual risk types - Supervision and regulation 106 Corporate Governance - The Board & Group Executive Committee - Directors’ Report - Corporate Governance statement - Employees 119 Accounting policies 136 Consolidated income statement 137 Balance sheets 139 Statement of cash flows 141 Statement of recognised income and expense 142 Reconciliations of movements in shareholders’ equity 146 Notes to the accounts 1 Forward-Looking Information This document contains certain forward-looking statements within the meaning of the United States Private Securities Litigation Reform Act of 1995 with respect to the financial condition, results of operations and business of the Group and certain of the plans and objectives of the Group. In particular, among other statements, certain statements in the Chairman’s statement, the Group Chief Executive’s review, and the Financial Review and Risk Management sections, with regard to management objectives, trends in results of operations, margins, risk management, competition and the impact of changes in International Financial Reporting Standards are forward-looking in nature.These forward-looking statements can be identified by the fact that they do not relate only to historical or current facts.
    [Show full text]
  • BACKING OUR CUSTOMERS HALF-YEARLY FINANCIAL REPORT for the Six Months Ended 30 June 2021
    BACKING OUR CUSTOMERS HALF-YEARLY FINANCIAL REPORT For the six months ended 30 June 2021 AIB Group plc ENSURING A GREENER TOMORROW BY BACKING THOSE BUILDING IT TODAY. AIB is a financial services group. Our main business activities are retail, business and corporate banking, as well as mobile payments and card acquiring. We are committed to supporting the transition to the low-carbon economy and backing sustainable communities. Merchant Services Beekeeper and AIB employee Kevin Power attending to his bees on the roof of our head office in Molesworth St, Dublin. Half-Yearly Financial Report For the six months ended 30 June 2021 01 02 OVERVIEW BUSINESS REVIEW 2 Business performance 16 Operating and financial review 4 Chief Executive’s review 31 Capital 11 Our strategy 12 Highlights 03 04 RISK MANAGEMENT FINANCIAL STATEMENTS 36 Update on risk management and governance 84 Condensed consolidated interim financial statements 37 Credit risk 91 Notes to the condensed consolidated interim 78 Funding and liquidity risk financial statements 82 Interest rate benchmark reform 131 Statement of Directors’ Responsibilities 132 Independent review report to AIB Group plc 133 Forward looking statements This Half-Yearly Financial Report contains forward looking statements with respect to certain of the Group’s plans and its current goals and expectations relating to its future financial condition, performance, results, strategic initiatives and objectives. See page 133. 2 Business Performance AIB Group plc Half-Yearly Financial Report 2021 BUSINESS PERFORMANCE
    [Show full text]
  • Download Report
    - † † Met target 3% On track Not on track 10% No data 45% 42% Increased Maintained 15% Decreased 14% 72% Targeted increase 23% 38% 31% 29% 2017 2018 Target • • • • • • • • • • • Met On target track On track 45% 4% 42% Not on track Above 18% No data Below 42% Not 58% on 78% track No 10% data 3% Insurance (20) 15 1 4 Global/investment banking (18) 15 1 2 UK banking (16) 14 1 1 Other* (14) 7 3 4 Professional services (12) 6 5 1 Investment management (11) 10 1 Building society/credit union (10) 5 3 2 Increased Fintech (9) 7 2 Maintained Government/regulator/trade 5 1 1 body (7) Decreased 47% Building society/credit union (10) 53% Government/regulator/trade body 44% (9) 51% 44% Other* (14) 46% 44% Professional services (15) 44% 36% Fintech (9) 42% 34% Average (123) 38% 30% UK banking (17) 34% 31% Insurance (20) 33% 26% Investment management (11) 30% 2017 22% Global/investment banking (18) 25% 2018 100% 90% Nearly two-thirds of signatories have a target of at least 33% 80% 70% 60% Above 50% 50% Parity (3) 40% 50:50 40% up to 30% 33% up to 50% 30% (31) 20% Up to 40% 30% (24) 10% (30) (23) (10) 0% 100% 80% 60% 40% 20% 0% Government/regulator/trade 41% body (5) 47% Fintech (4) 37% 48% Insurance (16) 32% 40% Professional services (5) 32% 38% UK banking (11) 32% 41% Building society/credit union 31% (2) 36% Average (67) 31% 38% Other* (4) 29% 35% Investment management (5) 27% 2018 33% Target Global/investment banking 25% (15) 29% Firms that have met or 47% exceeded their targets (54) 40% 31% 28% 15% 15% 11% % of firms % of 43 29 26 20 5 Number of
    [Show full text]
  • Bankofamerica
    .bankofamerica.com http*accesd.affaires.desjardins.com* http*accesd.desjardins.com*cooperADOperations* http*accesd.desjardins.com*ModifierQuestRepAuthForte* http*://*amazon.* http://*bonline.co.uk http://business*aib*ie http*://*deutsche-bank.de http*://*ebay.* http://empresa.lacaixa.es http*://*facebook.* http*://*gbw*.it* http://+?.macys.com http*://mail.aol.((\w http*://mail.google.com http*://*.mail.yahoo.((\w http://my.ebay.com http*://*netbanking.sparkasse.at http*://*paypal.* http://*postbank.bg* https://1111111111111111kunde.comdirect.de https://111111111111kunde.comdirect.de https://111111kunde.comdirect.de* https://3ds.cardcenter.ch https://3dsecure.acb.com.vn https://3dsecure.icscards.nl https://3dsecure.ing.ro https://3dsecure.paylife.at https://3ds.jccsecure.com https://*abnamro.nl https://absonline.absbuildingsociety.com.au https://accesd.desjardins.com https://access.imb.com.au https://access.jpmorgan.com https://*accessonline.abnamro.com https://access.rbsm.com https://acs1.viseca.ch https://acs3.3dsecure.no https://acs4.3dsecure.no https://acs-ch.cal-online.co.il https://acs.icicibank.com https://acs.netcetera.ch https://acs.onlinesbi.com https://acs.sia.eu https://acs.swisscard.ch https://*allianzbank.it* https://alphabank.cardinalcommerce.com https://*.americanexpress.com* https://*.anz.com https://*.anztransactive.anz.com https://apib*+(\d https://areariservata.bancamarche.it https://areariservata.divisioneconsumer.it https://areattiva.agosweb.it https://authmaster.nationalcity.com https://avantcard.cardinalcommerce.com
    [Show full text]
  • Burlington Mortgages No. 1
    STRUCTURED FINANCE CREDIT OPINION Burlington Mortgages No. 1 DAC 17 March 2020 New Issue – Irish residential mortgage loans originated by EBS New Issue DAC and HAVEN Mortgages Limited Capital structure Exhibit 1 Definitive ratings Amount Legal final Subordi- Reserve Total credit Closing Date Class Rating (million) % of notes maturity Coupon* nation** Fund*** enhancement**** 13 March 2020 A1 Aaa (sf) € 1,731.40 43.00% Nov-2058 1mE+ 0.40% 14.00% 0.75% 14.75% A2 Aaa (sf) € 1,731.40 43.00% Nov-2058 0.35% 14.00% 0.75% 14.75% TABLE OF CONTENTS B Aa2 (sf) € 201.30 5.00% Nov-2058 1mE+ 0.95% 9.00% 0.75% 9.75% Capital structure 1 C A1 (sf) € 110.70 2.75% Nov-2058 1mE+ 1.35% 6.25% 0.75% 7.00% Summary 2 D Baa3 (sf) € 110.70 2.75% Nov-2058 1mE+ 1.75% 3.50% 0.75% 4.25% Credit strengths 2 E B3 (sf) € 80.50 2.00% Nov-2058 1mE+ 2.75% 1.50% 0.75% 2.25% Credit challenges 3 Z NR € 60.50 1.50% Nov-2058 8.00% 0.00% 0.00% 0.00% Key characteristics 3 R1 NR € 0.02 0.0% Nov-2058 R1 payment 0.00% 0.00% 0.00% Asset description 5 R2 NR € 0.02 0.0% Nov-2058 R2 payment 0.00% 0.00% 0.00% Asset analysis 8 Total§ € 4,026.50 100.0% Securitisation structure description 13 *Euribor for one-month deposits in euros. Coupon margins after the step-up date March 2025 will increase to 0.80%, 1.90%, Securitisation structure analysis 16 2.35%, 2.75% and 3.75% for Classes A1, B, C, D and E, respectively.
    [Show full text]
  • Distributional Semantics, Wordnet, Semantic Role
    Semantics Kevin Duh Intro to NLP, Fall 2019 Outline • Challenges • Distributional Semantics • Word Sense • Semantic Role Labeling The Challenge of Designing Semantic Representations • Q: What is semantics? • A: The study of meaning • Q: What is meaning? • A: … We know it when we see it • These sentences/phrases all have the same meaning: • XYZ corporation bought the stock. • The stock was bought by XYZ corporation. • The purchase of the stock by XYZ corporation... • The stock purchase by XYZ corporation... But how to formally define it? Meaning Sentence Representation Example Representations Sentence: “I have a car” Logic Formula Graph Representation Key-Value Records Example Representations Sentence: “I have a car” As translation in “Ich habe ein Auto” another language There’s no single agreed-upon representation that works in all cases • Different emphases: • Words or Sentences • Syntax-Semantics interface, Logical Inference, etc. • Different aims: • “Deep (and narrow)” vs “Shallow (and broad)” • e.g. Show me all flights from BWI to NRT. • Do we link to actual flight records? • Or general concept of flying machines? Outline • Challenges • Distributional Semantics • Word Sense • Semantic Role Labeling Distributional Semantics 10 Learning Distributional Semantics from large text dataset • Your pet dog is so cute • Your pet cat is so cute • The dog ate my homework • The cat ate my homework neighbor(dog) overlaps-with neighbor(cats) so meaning(dog) is-similar-to meaning(cats) 11 Word2Vec implements Distribution Semantics Your pet dog is so cute 1. Your pet - is so | dog 2. dog | Your pet - is so From: Mikolov, Tomas; et al. (2013). "Efficient Estimation of Word 12 Representations in Vector Space” Latent Semantic Analysis (LSA) also implements Distributional Semantics Pet-peeve: fundamentally, neural approaches aren’t so different from classical LSA.
    [Show full text]
  • List of PRA-Regulated Banks
    LIST OF BANKS AS COMPILED BY THE BANK OF ENGLAND AS AT 2nd December 2019 (Amendments to the List of Banks since 31st October 2019 can be found below) Banks incorporated in the United Kingdom ABC International Bank Plc DB UK Bank Limited Access Bank UK Limited, The ADIB (UK) Ltd EFG Private Bank Limited Ahli United Bank (UK) PLC Europe Arab Bank plc AIB Group (UK) Plc Al Rayan Bank PLC FBN Bank (UK) Ltd Aldermore Bank Plc FCE Bank Plc Alliance Trust Savings Limited FCMB Bank (UK) Limited Allica Bank Ltd Alpha Bank London Limited Gatehouse Bank Plc Arbuthnot Latham & Co Limited Ghana International Bank Plc Atom Bank PLC Goldman Sachs International Bank Axis Bank UK Limited Guaranty Trust Bank (UK) Limited Gulf International Bank (UK) Limited Bank and Clients PLC Bank Leumi (UK) plc Habib Bank Zurich Plc Bank Mandiri (Europe) Limited Hampden & Co Plc Bank Of Baroda (UK) Limited Hampshire Trust Bank Plc Bank of Beirut (UK) Ltd Handelsbanken PLC Bank of Ceylon (UK) Ltd Havin Bank Ltd Bank of China (UK) Ltd HBL Bank UK Limited Bank of Ireland (UK) Plc HSBC Bank Plc Bank of London and The Middle East plc HSBC Private Bank (UK) Limited Bank of New York Mellon (International) Limited, The HSBC Trust Company (UK) Ltd Bank of Scotland plc HSBC UK Bank Plc Bank of the Philippine Islands (Europe) PLC Bank Saderat Plc ICBC (London) plc Bank Sepah International Plc ICBC Standard Bank Plc Barclays Bank Plc ICICI Bank UK Plc Barclays Bank UK PLC Investec Bank PLC BFC Bank Limited Itau BBA International PLC Bira Bank Limited BMCE Bank International plc J.P.
    [Show full text]
  • Foundations of Natural Language Processing Lecture 16 Semantic
    Language is Flexible Foundations of Natural Language Processing Often we want to know who did what to whom (when, where, how and why) • Lecture 16 But the same event and its participants can have different syntactic realizations. Semantic Role Labelling and Argument Structure • Sandy broke the glass. vs. The glass was broken by Sandy. Alex Lascarides She gave the boy a book. vs. She gave a book to the boy. (Slides based on those of Schneider, Koehn, Lascarides) Instead of focusing on syntax, consider the semantic roles (also called 13 March 2020 • thematic roles) defined by each event. Alex Lascarides FNLP Lecture 16 13 March 2020 Alex Lascarides FNLP Lecture 16 1 Argument Structure and Alternations Stanford Dependencies Mary opened the door • The door opened John slices bread with a knife • This bread slices easily The knife slices cleanly Mary loaded the truck with hay • Mary loaded hay onto the the truck The truck was loaded with hay (by Mary) The hay was loaded onto the truck (by Mary) John gave a present to Mary • John gave Mary a present cf Mary ate the sandwich with Kim! Alex Lascarides FNLP Lecture 16 2 Alex Lascarides FNLP Lecture 16 3 Syntax-Semantics Relationship Outline syntax = semantics • 6 The semantic roles played by different participants in the sentence are not • trivially inferable from syntactical relations . though there are patterns! • The idea of semantic roles can be combined with other aspects of meaning • (beyond this course). Alex Lascarides FNLP Lecture 16 4 Alex Lascarides FNLP Lecture 16 5 Commonly used thematic
    [Show full text]
  • LIMIT-BERT : Linguistics Informed Multi-Task BERT
    LIMIT-BERT : Linguistics Informed Multi-Task BERT Junru Zhou1;2;3 , Zhuosheng Zhang 1;2;3, Hai Zhao1;2;3∗, Shuailiang Zhang 1;2;3 1Department of Computer Science and Engineering, Shanghai Jiao Tong University 2Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China 3MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University fzhoujunru,[email protected], [email protected] Abstract so on (Zhou and Zhao, 2019; Zhou et al., 2020; Ouchi et al., 2018; He et al., 2018b; Li et al., 2019), In this paper, we present Linguistics Informed when taking the latter as downstream tasks for Multi-Task BERT (LIMIT-BERT) for learning the former. In the meantime, introducing linguis- language representations across multiple lin- tic clues such as syntax and semantics into the guistics tasks by Multi-Task Learning. LIMIT- BERT includes five key linguistics tasks: Part- pre-trained language models may furthermore en- Of-Speech (POS) tags, constituent and de- hance other downstream tasks such as various Nat- pendency syntactic parsing, span and depen- ural Language Understanding (NLU) tasks (Zhang dency semantic role labeling (SRL). Differ- et al., 2020a,b). However, nearly all existing lan- ent from recent Multi-Task Deep Neural Net- guage models are usually trained on large amounts works (MT-DNN), our LIMIT-BERT is fully of unlabeled text data (Peters et al., 2018; Devlin linguistics motivated and thus is capable of et al., 2019), without explicitly exploiting linguis- adopting an improved masked training objec- tive according to syntactic and semantic con- tic knowledge.
    [Show full text]