DOCSLIB.ORG

Dynabench: Rethinking Benchmarking in NLP Douwe Kiela†, Max Bartolo‡, Yixin Nie?, Divyansh Kaushik§, Atticus Geiger¶

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Dynabench: Rethinking Benchmarking in NLP Douwe Kiela†, Max Bartolo‡, Yixin Nie?, Divyansh Kaushik§, Atticus Geiger¶ Dynabench: Rethinking Benchmarking in NLP Douwe Kielay, Max Bartoloz, Yixin Nie?, Divyansh Kaushikx, Atticus Geiger{, Zhengxuan Wu{, Bertie Vidgenk, Grusha Prasad??, Amanpreet Singhy, Pratik Ringshiay, Zhiyi May, Tristan Thrushy, Sebastian Riedelyz, Zeerak Waseemyy, Pontus Stenetorpz, Robin Jiay, Mohit Bansal?, Christopher Potts{ and Adina Williamsy y Facebook AI Research; z UCL; ? UNC Chapel Hill; x CMU; { Stanford University k Alan Turing Institute; ?? JHU; yy Simon Fraser University [email protected] Abstract 0.2 MNIST ImageNet SQuAD 2.0 We introduce Dynabench, an open-source plat- GLUE SQuAD 1.1 Switchboard form for dynamic dataset creation and model 0.0 benchmarking. Dynabench runs in a web 0.2 browser and supports human-and-model-in- the-loop dataset creation: annotators seek to 0.4 create examples that a target model will mis- 0.6 classify, but that another person will not. In this paper, we argue that Dynabench addresses 0.8 a critical need in our community: contempo- rary models quickly achieve outstanding per- 1.0 formance on benchmark tasks but nonethe- 2000 2005 2010 2015 2020 less fail on simple challenge examples and falter in real-world scenarios. With Dyn- Figure 1: Benchmark saturation over time for popular abench, dataset creation, model development, benchmarks, normalized with initial performance at mi- and model assessment can directly inform nus one and human performance at zero. each other, leading to more robust and infor- mative benchmarks. We report on four ini- tial NLP tasks, illustrating these concepts and are part of the reason we can confidently say we highlighting the promise of the platform, and have made great strides in our field. address potential objections to dynamic bench- In light of these developments, one might be marking as a new standard for the field. forgiven for thinking that NLP has created mod- els with human-like language capabilities. Prac- 1 Introduction titioners know that, despite our progress, we are While it used to take decades for machine learning actually far from this goal. Models that achieve models to surpass estimates of human performance super-human performance on benchmark tasks (ac- on benchmark tasks, that milestone is now rou- cording to the narrow criteria used to define hu- tinely reached within just a few years for newer man performance) nonetheless fail on simple chal- arXiv:2104.14337v1 [cs.CL] 7 Apr 2021 datasets (see Figure1). As with the rest of AI, NLP lenge examples and falter in real-world scenarios. has advanced rapidly thanks to improvements in A substantial part of the problem is that our bench- computational power, as well as algorithmic break- mark tasks are not adequate proxies for the so- throughs, ranging from attention mechanisms (Bah- phisticated and wide-ranging capabilities we are danau et al., 2014; Luong et al., 2015), to Trans- targeting: they contain inadvertent and unwanted formers (Vaswani et al., 2017), to pre-trained lan- statistical and social biases that make them artifi- guage models (Howard and Ruder, 2018; Devlin cially easy and misaligned with our true goals. et al., 2019; Liu et al., 2019b; Radford et al., 2019; We believe the time is ripe to radically rethink Brown et al., 2020). Equally important has been the benchmarking. In this paper, which both takes a rise of benchmarks that support the development of position and seeks to offer a partial solution, we ambitious new data-driven models and that encour- introduce Dynabench, an open-source, web-based age apples-to-apples model comparisons. Bench- research platform for dynamic data collection and marks provide a north star goal for researchers, and model benchmarking. The guiding hypothesis be- hind Dynabench is that we can make even faster of much recent interest. Challenging test sets have progress if we evaluate models and collect data shown that many state-of-the-art NLP models strug- dynamically, with humans and models in the loop, gle with compositionality (Nie et al., 2019; Kim rather than the traditional static way. and Linzen, 2020; Yu and Ettinger, 2020; White Concretely, Dynabench hosts tasks for which et al., 2020), and find it difficult to pass the myriad we dynamically collect data against state-of-the- stress tests for social (Rudinger et al., 2018; May art models in the loop, over multiple rounds. The et al., 2019; Nangia et al., 2020) and/or linguistic stronger the models are and the fewer weaknesses competencies (Geiger et al., 2018; Naik et al., 2018; they have, the lower their error rate will be when in- Glockner et al., 2018; White et al., 2018; Warstadt teracting with humans, giving us a concrete metric— et al., 2019; Gauthier et al., 2020; Hossain et al., i.e., how well do AI systems perform when inter- 2020; Jeretic et al., 2020; Lewis et al., 2020; Saha acting with humans? This reveals the shortcomings et al., 2020; Schuster et al., 2020; Sugawara et al., of state-of-the-art models, and it yields valuable 2020; Warstadt et al., 2020). Yet, challenge sets training and assessment data which the community may suffer from performance instability (Liu et al., can use to develop even stronger models. 2019a; Rozen et al., 2019; Zhou et al., 2020) and In this paper, we first document the background often lack sufficient statistical power (Card et al., that led us to propose this platform. We then de- 2020), suggesting that, although they may be valu- scribe the platform in technical detail, report on able assessment tools, they are not sufficient for findings for four initial tasks, and address possible ensuring that our models have achieved the learn- objections. We finish with a discussion of future ing targets we set for them. plans and next steps. Models are susceptible to adversarial attacks, and despite impressive task-level performance, 2 Background state-of-the-art systems still struggle to learn robust Progress in NLP has traditionally been measured representations of linguistic knowledge (Ettinger through a selection of task-level datasets that grad- et al., 2017), as also shown by work analyzing ually became accepted benchmarks (Marcus et al., model diagnostics (Ettinger, 2020; Ribeiro et al., 1993; Pradhan et al., 2012). Recent well-known 2020). For example, question answering models examples include the Stanford Sentiment Tree- can be fooled by simply adding a relevant sentence bank (Socher et al., 2013), SQuAD (Rajpurkar to the passage (Jia and Liang, 2017). et al., 2016, 2018), SNLI (Bowman et al., 2015), Text classification models have been shown to be and MultiNLI (Williams et al., 2018). More sensitive to single input character change (Ebrahimi recently, multi-task benchmarks such as SentE- et al., 2018b) and first-order logic inconsisten- val (Conneau and Kiela, 2018), DecaNLP (McCann cies (Minervini and Riedel, 2018). Similarly, ma- et al., 2018), GLUE (Wang et al., 2018), and Super- chine translation systems have been found suscepti- GLUE (Wang et al., 2019) were proposed with the ble to character-level perturbations (Ebrahimi et al., aim of measuring general progress across several 2018a) and synthetic and natural noise (Belinkov tasks. When the GLUE dataset was introduced, and Bisk, 2018; Khayrallah and Koehn, 2018). Nat- “solving GLUE” was deemed “beyond the capabil- ural language inference models can be fooled by ity of current transfer learning methods” (Wang simple syntactic heuristics or hypothesis-only bi- et al., 2018). However, GLUE saturated within a ases (Gururangan et al., 2018; Poliak et al., 2018; year and its successor, SuperGLUE, already has Tsuchiya, 2018; Belinkov et al., 2019; McCoy et al., models rather than humans at the top of its leader- 2019). Dialogue models may ignore perturbations board. These are remarkable achievements, but of dialogue history (Sankar et al., 2019). More there is an extensive body of evidence indicating generally, Wallace et al.(2019) find universal ad- that these models do not in fact have the human- versarial perturbations forcing targeted model er- level natural language capabilities one might be rors across a range of tasks. Recent work has also lead to believe. focused on evaluating model diagnostics through counterfactual augmentation (Kaushik et al., 2020), 2.1 Challenge Sets and Adversarial Settings decision boundary analysis (Gardner et al., 2020; Whether our models have learned to solve tasks Swayamdipta et al., 2020), and behavioural test- in robust and generalizable ways has been a topic ing (Ribeiro et al., 2020). 2.2 Adversarial Training and Testing 3 Dynabench Research progress has traditionally been driven by Dynabench is a platform that encompasses different a cyclical process of resource collection and ar- tasks. Data for each task is collected over multiple chitectural improvements. Similar to Dynabench, rounds, each starting from the current state of the recent work seeks to embrace this phenomenon, ad- art. In every round, we have one or more target dressing many of the previously mentioned issues models “in the loop.” These models interact with through an iterative human-and-model-in-the-loop humans, be they expert linguists or crowdworkers, annotation process (Yang et al., 2017; Dinan et al., who are in a position to identify models’ shortcom- 2019; Chen et al., 2019; Bartolo et al., 2020; Nie ings by providing examples for an optional context. et al., 2020), to find “unknown unknowns” (Atten- Examples that models get wrong, or struggle with, berg et al., 2015) or in a never-ending or life-long can be validated by other humans to ensure their learning setting (Silver et al., 2013; Mitchell et al., correctness. The data collected through this pro- 2018). The Adversarial NLI (ANLI) dataset (Nie cess can be used to evaluate state-of-the-art models, et al., 2020), for example, was collected with an and to train even stronger ones, hopefully creat- adversarial setting over multiple rounds to yield ing a virtuous cycle that helps drive progress in “a ‘moving post’ dynamic target for NLU systems, the field.
Load more

Recommended publications
  • The Letters of Marcus Tullius Cicero to Several of His Friends
    UC SOUTHERN REGIONAL LIBRARY FACILITY AA 000 464 615 4 Ex Libris C. K. OGDEN w m '»? ^ .'. -vX , \ > • I LETTERTHE OF Marcus Tulliiis Cicero T O Several of his Friends: With REMARKS By W I L L I A M M E L M O T H, Efqi Quo fit ut omnis Votlva pateat veluti defcripta tabella Vita fenis. HoR. THE THIRD EDITION. IN THREE VOLUMES. VOL. III. i:x.:\/usfj{eg. 6V^//. LONDON: Printed for J. Dodsley, in Pall-mall. 177J, SRLF URl fft LETTERS.,...— , E^-nij OF Marcus Tullius Cicero T O Several of his F R i E N d So BOOK XI. LETTER L To Tiro. letter encourages me to hope that A.u.70?. find better : I am fure at YOURyou yourfelf leaftj I moil fincerely wifh that you may. I intreat you therefore to confecrate all your cares to that end ; and by no means indulge fo miftaken a fufpicion as that I am difpleafed you are not with me. With me you are, in the bell fenfe of that exprelTion, if you are tak- ing care of your health : which I had much rather you Ihouid attend^ than on myfelf. For Vol. UI. B tho' 2 The letters Book XL .v.u.7o8.tho'' I always both fee and hear you with plea- lure that will j pleafure be greatly increafed, when I fhali have the fatisfaftiory at the fame tii-n* to be afiured that you are perfectly well. My work is at prefcnt fufpended \ as I can- not make ufe of my own hand: however I em- ploy myfclf a good deal in reading.
    [Show full text]
  • The History of the Celtic Language May Be Turned To
    '^^'& msw 6iW. l(o?^ )^. HISTORY CELTIC LANGUAGE; WHEKEIX IT IS SHOWN TO BE BASED UPON NATURAL PRINCIPLES, AXD, ELEMENTARILY CONSIDERED, CONTEMPORANEOUS WITH THE INFANCY OF THE HUMAN FAMILY : LIZEWISE SHOWING ITS IMPORTANCE IN ORDER TO THE PROPER UNDERSTANDING OF THE CLASSICS, INCLUDING THE SACRED TEXT, THE HIEROGLYPHICS, THE CABALA, ETC. ETC. BY L. MACLEAN, F.O.S, kuthnr of" Historical Account of lona," " Sketches of St Kilda," &c. Sec. LONDON: SMITH, ELDER, and CO.; EDINBURGH: M'LACHLAN, STEWART, and CO. GLASGOW: DUGALD MOORE. MDCCCXL. " IT CONTAINS MANY TRUTHS WHICH ARE ASTOUNDING, AND AT WHICH THE IGNORANT MAY SNEER; BUT THAT WILL NOT TAKE PROM THEIR ACCURACY. "_SEB SIR WILLIAM BETHAM's LETTER TO THE AUTHOR IN REFERENCE TO THE GAELIC EDITION. " WORDS ARE THE DAUGHTERS OF EARTH—THINGS ARE THE SONS OF HEAVEN."—SAMUEL JOHNSON, GLASGOW: — F.nWAKi) KHII.I., I'Hl NTER TO THE U M VERSITV. ^' D IBtKication^ RIGHT HONOURABLE SIR ROBERT PEEL, baronet, m.p. Sir, An ardent admirer of your character, public and private, I feel proud of the permission you have kindly granted me to Dedicate to you this humble Work. The highest and most noble privilege of great men is the opportunity their station affords them of fostering the Fine Arts, and amplifying the boundaries of useful knowledge. That this spirit animates your bosom, each successive day is adding proof: nor is the fact IV DEDICATION. unknown, that whilst your breast glows with the fire of the patriot, beautifully harmonizing with the taste of the scholar, your energies are likewise engaged on the side of that pure religion of your fathers, with which your own mind has been so early imbued, and which, joined with Education, is, as has properly been said, " the cheapest defence of a nation;" as it is the only solid foundation whereon to build our hopes of bliss in a world to come.
    [Show full text]
  • Studies in Tennyson Poems of Tennyson
    1920. COPTBIGHT, 1889. 1891. 1892. 1897, 1898. BY CHARLES SCRIBNEB's SONS Published February, 1920 PR. 558% V4 THE 8CRBNER PRESS BY HENRY VAN DYKE The Valley of Vision Fighting for Peace The Unknown Quantity The Ruling Passion The Blue Flower Out-of-Doors in the Holy Land Days Off Little Rivers Fisherman's Luck Poems, Collection in one volume Golden Stars The Red Flower The Grand Canyon, and Other Poems The White Bees, and Other Poems The Builders, and Other Poems Music, and Other Poems The Toiling of Felix, and Other Poems The House of Rimmon Studies in Tennyson Poems of Tennyson CHARLES SCRIBNER'S SONS STUDIES IN TENNYSON <J / A YOUNG WOMAN OF AN OLD FASHION WHO LOVES ABT NOT ONLY FOE ITS OWN SAKE BUT BECAUSE IT ENNOBLES LIFE WHO READS POETRY NOT TO KILL TIME BUT TO FILL IT WITH BEAUTIFUL THOUGHTS AND WHO STILL BELIEVES IN GOD AND DUTY AND IMMORTAL LOVE I DEDICATE THIS BOOK PREFACE 1 HIS volume is intended to be a companion to my Select Poems of Tennyson. I have put it second in the pair because that is its right place. Criticisms, com^ ments, interpretations, are of comparatively little use until you have read the poetry of which they treat. Like photographs of places that one has not seen, they lack the reviving, realizing touch of remembrance. The book contains a series of essays, written at dif- ferent times, printed separately in different places, and collected, substantially, in a book called The Poetry of Tennyson, which was fortunate enough to find many friends, and has now, I believe, gone out of print.
    [Show full text]
  • Fishes of the Philippine Islands
    FISHES OF THE PIDLIPPINE ISLANDS. By BARTON WARREN EVERMANN. A. M••Ph.D••and ALVIN SEALE. A. B. BUREAU OF FISHERIES DOCUMENT NO. 607. B. B. F. i906-4 49 Blank page retained for pagination FISHES OF THE PHILIPPINE ISLANDS. By BARTON WARREN EVERMANN, A. M., PH. D., AND ALVIN SEAJ~E, A. B. Among the exhibits made by the U. S. Bureau of Insular Affairs at the Lou­ isiana Purchase Exposition, held at St. Louis, Missouri, during the summer and fall of 1904, was a considerable collection of fishes from the Philippine Islands. This collection had been made under the direction of the Philippine government for the specific purpose mentioned. At the close of the exposition the authorities turned the entire collection over.to the U. S. Bureau of Fisheries for study and report. The localities represented in the collection are as follows: San Fabian, Pangasinan, Luzon.~San Fabian is situated on the Gulf of Lin­ gayen, about 100 miles north of Manila. The collection from this place was made by Mr. C. L. Hall and is a-large and important one. Bacon, Sorsoqon, Luzon.-Sorsogon is the most southern province of Luzon. Bacon is situated in the northern part of the province, on Albay Inlet, in approxi- mately latitude 130 3' north and longitude 1240 east. Bulan, Sorsogon, Luzon.-Bulan is on the west coast of Sorsogon, near the southern end of the province, approximately in latitude 120 38' north, longitude 1230 22' east. The collections from Bacon and Bulan are more extensive than any of the others.
    [Show full text]
  • Plainheld 12 Pages Nrs.Sperry Is 96
    {FEATHER FORECAST March K. 1*17 12 PAGES Ttr»IUr toalshttonigh! anaand Suaday.Sund V- KKMPBRATCRB IN WHICH IS MERGED THE AVKRAfif!LVKHAKK DAILYDAILY CIRCU-I'llll C- k LATIONLATION LASTLAST WEEKWEKK IdaBigbet**<t yaslardayyesterday . ..51*6 bowertLowest thisthis mommornin leag .....41..43 PLAINHELPLAINFIELDD DAILY PRESS, 7,787,7899 I’LIIM!•!. IliniU IEUI,l JEIKWW JKRHKY.JKKSKV. UI11UIAT,MI1HIUI, MARCHJHBtH MM,, 1917J»1T.. PRICEPRICE TWOTWO CENTSOEHTS DR.STR0NGTO SPEAK C. A. STEINER NRS.SPERRHRS. SPERRYY HUMARUHANN SIDSIDEE OFOF DR. STRONGTO SPEAK C. A. STEINER CHURCCHURCHH SOCIETYSOCIETY MYSTERIOUMYSTERIOUSS pANOEUVRINMANOEUVRINGG OFFOFF FOFORR JEWISHJEWISH RELIEFRELIEF DIEDIESS ININ WESTWEST 25 YEARS OLD B 96 YEARS OLD TRENCTRENCHH WAWARR 25 YEARS OLD IS 96 YEARS OLD T~j— 1 • tHEM/||sAGBUSETTtHE MASSACHUSETTSS COASCOASTT WifWilll AddresiAddress thethe MeetinMeetingg iinn Jiforme r LocalLocal ResidenResidentt Ladies’ Aid Society of Bor- BByy UnitedUnited fresspress:: >’ i ~~^ Colombo*Columbus AuditoriumAuditorium oonn Fasses Away at Banning, Ladiesough' AiChorchd Societ Heldy Birtb-of Bor - Afed Borough Woman Cele- Captain Ian Hay Beilh Ad- Passes Away at Banning, Aged Borough W»nun Cele- Csptein Ian Hay Beith Ad- ApriAprill 33dd ough Church Held Birth- NewNew Bedford.Bedford, MarchM$j3l 24.—Mysteriou24 — Mysteriouss manoeuvring brates Event at Home of dressed Big Audience Under CaL—FuneralCaL—Funeral HereHere day Party by the watbr-front poltce, and- investigations of Brandt's bntet Event at Home of irtnei Big Audience Under Dr Philip D. Strong. of tb# Pint Word of iha daath or durl Alban '■ Tha birthdayday Partparly y cl--a la Daughter,Daughter, Mr*.Mrs. Steren*Stevens National Special Aid Baptist Church, ha*. —oonaented to.... Stainer. a(a 30 year.
    [Show full text]
  • Journal of Roman Studies Roman Inscriptions 2006–2010
    Journal of Roman Studies http://journals.cambridge.org/JRS Additional services for Journal of Roman Studies: Email alerts: Click here Subscriptions: Click here Commercial reprints: Click here Terms of use : Click here Roman Inscriptions 2006–2010 Alison E. Cooley and Benet Salway Journal of Roman Studies / Volume 102 / November 2012, pp 172 ­ 286 DOI: 10.1017/S0075435812001074, Published online: 01 October 2012 Link to this article: http://journals.cambridge.org/abstract_S0075435812001074 How to cite this article: Alison E. Cooley and Benet Salway (2012). Roman Inscriptions 2006–2010. Journal of Roman Studies, 102, pp 172­286 doi:10.1017/S0075435812001074 Request Permissions : Click here Downloaded from http://journals.cambridge.org/JRS, IP address: 144.82.107.89 on 05 Nov 2012 SURVEY ARTICLE Roman Inscriptions 2006–2010 ALISON E. COOLEY AND BENET SALWAY IGENERAL I.i General Introduction The aim of this quinquennial survey remains the same as its predecessor, as for the most part does the format, though the team is regrettably reduced by one.1 With an eye to the study of the Roman world, we hope to signal the most important newly published inscriptions, signicant reinterpretations of previously published material, new trends in scholarship, recent studies that draw heavily on epigraphic sources, and noteworthy developments in the various aids to understanding inscriptions (both traditional printed material and electronic resources). In the context of this journal, the geographical range and chronological scope reect the contours and history of the Roman state from its beginnings down to the end of the seventh century. As such, not only does the survey naturally take in Greek as well as Latin texts, but also epigraphic material in other languages relevant to the Roman world.
    [Show full text]
  • ATLAS of CLASSICAL HISTORY
    ATLAS of CLASSICAL HISTORY EDITED BY RICHARD J.A.TALBERT London and New York First published 1985 by Croom Helm Ltd Routledge is an imprint of the Taylor & Francis Group This edition published in the Taylor & Francis e-Library, 2003. © 1985 Richard J.A.Talbert and contributors All rights reserved. No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. British Library Cataloguing in Publication Data Atlas of classical history. 1. History, Ancient—Maps I. Talbert, Richard J.A. 911.3 G3201.S2 ISBN 0-203-40535-8 Master e-book ISBN ISBN 0-203-71359-1 (Adobe eReader Format) ISBN 0-415-03463-9 (pbk) Library of Congress Cataloguing in Publication Data Also available CONTENTS Preface v Northern Greece, Macedonia and Thrace 32 Contributors vi The Eastern Aegean and the Asia Minor Equivalent Measurements vi Hinterland 33 Attica 34–5, 181 Maps: map and text page reference placed first, Classical Athens 35–6, 181 further reading reference second Roman Athens 35–6, 181 Halicarnassus 36, 181 The Mediterranean World: Physical 1 Miletus 37, 181 The Aegean in the Bronze Age 2–5, 179 Priene 37, 181 Troy 3, 179 Greek Sicily 38–9, 181 Knossos 3, 179 Syracuse 39, 181 Minoan Crete 4–5, 179 Akragas 40, 181 Mycenae 5, 179 Cyrene 40, 182 Mycenaean Greece 4–6, 179 Olympia 41, 182 Mainland Greece in the Homeric Poems 7–8, Greek Dialects c.
    [Show full text]
  • Map 87 Pontus-Phasis Compiled by David Braund and T
    Map 87 Pontus-Phasis Compiled by David Braund and T. Sinclair (Turkey), 1997 with the assistance of Diane Braund Introduction Pontus The land of Pontus has two main parts, both of which belonged to the Mithridatic kingdom of Pontus in the Hellenistic period. The first is the main ridge of the Pontic mountains and the steep descent to the shore. The lower slopes are thickly wooded, and the shore districts have a humid and rainy climate. In the main Pontic ridge a gap occurs in the hinterland of Amisus (modern Samsun), after which the mountain chain continues at a lower height and with less abrupt slopes (Strabo’s term Paryadres seems to denote the higher part of the chain). As far east as Rhizaion (Rize) Greek settlements along the coast have existed since the sixth century B.C., and in the case of Sinope the seventh. To the south, the Pontic chain is bordered by Armenia Minor, a part of Armenia itself (Late Antique Sper, probably the Syspiritis of the classical authors), and what was to become the Georgian mountainous district of Tao, drained by the R. Glaukos (Oltu Çay) and the Tortum Çay. The district of Chaldia, the Gümüşhane basin, is difficult to classify, particularly in classical times, as there is no evidence that it belonged either to Pontus (in a geographical or administrative sense), to Armenia Minor, or to Armenia itself. The Roman empire made no effort to control the interior of Chaldia until the reign of Justinian. The second part of Pontus is the series of fertile inland plains, joined by relatively low chains of hills, in the river systems of the Iris (Kızıl Irmak) and the lower Halys (Yeşil Irmak).
    [Show full text]
  • 1 SATIRE in the HISTORIA AUGUSTA by SHAWN GAIUS
    SATIRE IN THE HISTORIA AUGUSTA By SHAWN GAIUS DANIELS A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2013 1 © 2013 Shawn Gaius Daniels 2 Matri meae 3 ACKNOWLEDGMENTS First, I would like to thank my advisor, Dr. Victoria Pagán. When I began to read the Historia Augusta in the Summer of 2010, I was struck most by its humor. Dr. Pagán recommended that I turn to Northrop Frye and his Anatomy of Criticism to find a theoretical basis on which to understand the role of humor, and it is there that I found a definition of satire that took me on the road to the present work. A large portion of the bibliography and the majority of the organizational structure came from the many conversations we held over the past three years. Without her unflagging support and invaluable advice, this project would not have been completed. I owe many thanks to Dr. Trevor Luke, of Florida State University. He found time, even at conferences, to speak with me individually about my research—once even driving down from Tallahassee to discuss a rough draft he had read the week before. His penetrating questions and moral support have kept me critical of my own work. I would also like to thank the other members of my committee, Dr. Andrew Wolpert, Dr. Kostas Kapparis, and Dr. Terry Harpold for sparing their time to help see that I accomplish this task. I would like to thank several former teachers and professors for helping bring me to this point.
    [Show full text]
  • Lipid Lowering Treatment in Mild to Moderate Aortic Stenosis Thesis for the Degree of Philosophiae Doctor (Phd)
    FACULTY OF MEDICINE, UNIVERSITY OF OSLO, AND DEPARTMENT OF CARDIOLOGY, OSLO UNIVERSITY HOSPITAL, AKER AND ULLEVÅL, OSLO, NORWAY 2015 Lipid Lowering Treatment in Mild to Moderate Aortic Stenosis Thesis for the Degree of philosophiae doctor (PhD) Anne Bjørhovde Rossebø December 2015 1 Thesis 2015. Anne Bjørhovde Rossebø © Anne Bjørhovde Rossebø, 2016 Series of dissertations submitted to the Faculty of Medicine, University of Oslo ISBN 978-82-8333-245-2 All rights reserved. No part of this publication may be reproduced or transmitted, in any form or by any means, without permission. Cover: Hanne Baadsgaard Utigard Printed in Norway: 07 Media AS – www.07.no CONTENTS ACKNOWLEDGEMENTS ........................................................................................................... 4 LIST OF PAPERS .......................................................................................................................... 7 Paper I .......................................................................................................................................... 7 Paper II......................................................................................................................................... 7 Paper III ....................................................................................................................................... 7 Paper IV ....................................................................................................................................... 7 ABBREVIATIONS .......................................................................................................................
    [Show full text]
  • Virtue from Necessity in the Urban Waterworks of Roman Asia Minor
    UNIVERSITY OF CALIFORNIA Santa Barbara Virtue from Necessity in the Urban Waterworks of Roman Asia Minor A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in History of Art and Architecture by Brianna Lynn Bricker Committee in charge: Professor Fikret K. Yegül, Chair Professor Swati Chattopadhyay Professor Christine M. Thomas June 2016 The dissertation of Brianna Lynn Bricker is approved: ____________________________________________ Swati Chattopadhyay ____________________________________________ Christine M. Thomas ____________________________________________ Fikret K. Yegül, Committee Chair June 2016 ACKNOWLEDGEMENTS Damlaya damlaya göl olur “Drop by drop, a lake will form,” goes the Turkish proverb. And so went this dissertation project: little by little quotidian efforts turned into something larger. But these drops, in fact, were not just my own efforts, nor were they always quotidian. The number of drops contributed by others is incalculable, but the impact is clear. Special thanks go to my advisor Fikret Yegül for his unwavering support and kindness throughout my graduate school experience. I also thank Diane Favro for her continual warmth and helpfulness. I am deeply thankful for my other committee members, Swati Chattopadhyay and Christine Thomas, for providing important outside perspectives and challenging me to go farther with my work. I am truly grateful for my community at UCSB: firstly, the Department of the History of Art and Architecture, for the support and space to grow as a scholar. I also thank the Ancient Borderlands Research Focus Group for fostering collaboration between disciplines and creating an inviting and dynamic atmosphere of research. I would particularly like to thank my community at the Sardis Expedition, for nurturing in me the curiosity and zeal to explore the Anatolian countryside: Marcus Rautman introduced me to the intricacies and interesting nature of waterworks, while Bahadır Yıldırım and Nicholas Cahill provided further encouragement and perspective.
    [Show full text]
  • Roman Roads Reader: Drama and Lyric
    Roman Roads Reader: Drama and Lyric Selections from Greek Drama & Lyric Companion Book for Greeks: Drama and Lyric, a video course by Roman Roads Media. Euripides, Sappho, Pindar, Hesiod, Theocritus, Quintus of Smyrna, and Apollonius of Rhodes Edited by Daniel Foucachon This book has been designed to accompany the video course Greeks: Drama and Lyric, part of the Old Western Culture series by Roman Roads Media. To find out more about this course, visit www.romanroadsmedia.com While the selections contained in this volume match the specific course of study for Drama and Lyric (mentioned above), this selection may be of great use and enjoyment to any student of the classics, and we hope it finds its way into your library. Other titles in the Old Western Culture Series by Roman Roads Media: Greeks: -- The Epics (The Iliad & The Odyssey) -- Drama and Lyric (The Tragedies, Comedies, and Minor Poets) -- The Histories (Herodotus, Thucydides, and Xenophon) -- The Philosophers (Plato and Aristotle) Romans: -- The Roman Epic (The Aeneid, Ovid, and Lucretius) -- The Historians (Livy, Tacitus, Salust, Julius Caesar, Plutarch, and Cicero) -- Early Christianity (Clementine, Ignatius, Irenaeus, Justin Martyr, and Eusebius) -- Post-Nicene Christianity (Athanasius, Augustine, and Boethius) Christendom: -- Early Medieval (St. Benedict, Bede, Charlemagne, and Alfred the Great) -- The Defense of the Faith (Anselm, Geffrey of Monmoth, The Golden Legend) -- The Medieval Mind (Dante and Aquinas) -- The Reformation (Erasmus, Calvin, Cranmer, Spencer, and Chaucer)
    [Show full text]