Proceedings of the First Italian Conference on Computational Linguistics CLiC-it 2014 Questo e-book appartiene a AlessandroLenci & the Fourth International Workshop EVALITA 2014 9-11 December 2014, Pisa Questo e-book appartiene a AlessandroLenci a appartiene e-book Questo Volume II Fourth International Workshop EVALITA 2014 Proceedings Editors Cristina Bosco, Piero Cosi, Felice Dell’Orletta, Mauro Falcone, Simonetta Montemagni, Maria Simi 11th December 2014 Pisa, Italy © Copyright 2014 by Pisa University Press srl Società con socio unico Università di Pisa Capitale Sociale Euro 20.000,00 i.v. - Partita IVA 02047370503 Sede legale: Lungarno Pacinotti 43/44 - 56126, Pisa Tel. + 39 050 2212056 Fax + 39 050 2212945 e-mail: [email protected] www.pisauniversitypress.it ISBN 978-886741-472-7 Questo e-book appartiene a AlessandroLenci 10.12871/clicit201420 Established in 2007, EVALITA (http://www.evalita.it) is the evaluation campaign of Natural Language Processing and Speech Technologies for the Italian language, organized around shared tasks focusing on the analysis of written and spoken language respectively. EVALITA’s shared tasks are aimed at contributing to the development and dissemination of natural language resources and technologies by proposing a shared context for training and evaluation. Following the success of previous editions, we organized EVALITA 2014, the fourth evaluation campaign with the aim of continuing to provide a forum for the comparison and evaluation of research outcomes as far as Italian is concerned from both academic institutions and industrial organizations. The event has been supported by the NLP Special Interest Group of the Italian Association for Artificial Intelligence (AI*IA) and by the Italian Association of Speech Science (AISV). The novelty of this year is that the final workshop of EVALITA is co-located with the 1st Italian Conference of Computational Linguistics (CLiC-it, http://clic.humnet.unipi.it/), a new event aiming to establish a reference forum for research on Computational Linguistics of the Italian community with contributions from a wide range of disciplines going from Computational Linguistics, Linguistics and Cognitive Science to Machine Learning, Computer Science, Knowledge Representation, Information Retrieval and Digital Humanities. The co-location with CLiC-it potentially widens the potential audience of EVALITA. The final workshop, held in Pisa on the 11th December 2014 within the context of the XIII AI*IA SymposiumQuesto e-book appartiene a AlessandroLenci on Artificial Intelligence (Pisa, 10-12 December 2014, http://aiia2014.di.unipi.it/), gathers the results of 8 tasks, 4 of which focusing on written language and 4 on speech technologies. In this EVALITA edition, we received 30 expressions of interest, 55 registrations and 43 actual submissions to 8 proposed tasks distributed as follows: • Written language tasks: Dependency Parsing - DP (5), Evaluation of Events and Temporal Information - EVENTI (6), Sentiment Polarity Classification - SENTIPOLC (27), Word Sense Disambiguation and Lexical Substitution - WSD&LS (0); • Speech tasks: Emotion Recogniton Task - ERT (2), Forced Alignment on Children Speech - FACS (1), Human and Machine Dialect Identification from Natural Speech and Artificial Stimuli - HMDI (0), Speech Activity Detection and Speaker Localization in Domestic Environments - SASLODOM (2). 23 participants (either as individual researchers or as academic institutions) submitted their results to one or more different tasks of the contest. In this volume, the reports of the tasks’ organizers and participants of EVALITA 2014 are collected. As in previous editions, both the tasks and the final workshop were collectively organized by several researchers from the community working on Italian language resources and technologies. We thank all the people and institutions involved in the organization of the tasks, who contributed to the success of the event. A special thank is due to Francesco Cutugno (Università Degli Studi di Napoli Federico II) for his important contribution to the organization of the EVALITA Speech tasks. Thanks are also due to Manuela Sanguinetti (Università di Torino) for helping with the management of the EVALITA website, and to FBK for making the web platform available for this III edition as well. Last but not least, we thank our invited speaker, Ryan McDonald from Google, for agreeing to share his expertise on key topics of EVALITA 2014. November 2014 EVALITA 2014 CO-CHAIRS Cristina Bosco Piero Cosi Felice Dell’Orletta Mauro Falcone Simonetta Montemagni Maria Simi Questo e-book appartiene a AlessandroLenci a appartiene e-book Questo IV EVALITA 2014 Scientific coordination • Cristina Bosco (Università di Torino) • Felice Dell'Orletta (Istituto di Linguistica Computazionale “Antonio Zampolli” - CNR, Pisa) • Simonetta Montemagni (Istituto di Linguistica Computazionale “Antonio Zampolli” - CNR, Pisa) • Maria Simi (Università di Pisa) EVALITA 2014 Scientific coordination for Speech Technology Evaluation • Piero Cosi (Istituto di Scienze e Tecnologie della Cognizione - CNR, Padova) • Mauro Falcone (Fondazione Ugo Bordoni) EVALITA 2014 Steering Committee Name Institution Task Valerio Basile University of Groningen, Netherlands SENTIPOLC Andrea Bolioli CELI, Torino, Italy SENTIPOLC Cristina Bosco Università di Torino, Italy DP Alessio Brutti Fondazione Bruno Kessler, Trento, Italy SASLODOM Tommaso Caselli VU Amsterdam, Netherlands EVENTI Piero Cosi Istituto di Scienze e Tecnologie della FACS Cognizione - CNR, Italy Francesco Cutugno Università di Napoli “Federico II”, Italy FACS Felice Dell’Orletta Istituto di Linguistica Computazionale DP “Antonio Zampolli” - CNR, Pisa Vincenzo Galatà AlessandroLenci a appartiene e-book Questo Istituto di Scienze e Tecnologie della ERT, FACS Cognizione - CNR, Italy Monica Monachini Istituto di Linguistica Computazionale EVENTI “Antonio Zampolli” - CNR, Pisa Simonetta Montemagni Istituto di Linguistica Computazionale DP “Antonio Zampolli” - CNR, Pisa Malvina Nissim University of Groningen and Università di SENTIPOLC Bologna, Netherlands, Italy Maurizio Omologo Fondazione Bruno Kessler, Trento, Italy SASLODOM Antonio Origlia Università di Napoli “Federico II”, Italy ERT, FACS Viviana Patti Università di Torino, Italy SENTIPOLC Mirco Ravanelli Fondazione Bruno Kessler, Trento, Italy SASLODOM Antonio Romano Università di Torino, Italy HDMI Paolo Rosso Universitat Politècnica de València, Spain SENTIPOLC Claudio Russo Università di Torino, Italy HDMI Manuela Sanguinetti Università di Torino, Italy DP Maria Simi Università di Pisa, Italy DP Manuela Speranza Fondazione Bruno Kessler, Trento, Italy EVENTI Rachele Sprugnoli Fondazione Bruno Kessler and University EVENTI of Trento, Trento, Italy V Indice AlessandroLenci a appartiene e-book Questo WRITTEN LANGUAGE TASKS Dependency Parsing The Evalita 2014 Dependency Parsing task Cristina Bosco, Felice Dell’Orletta, Simonetta Montemagni, Manuela Sanguinetti, Maria Simi 1 Dependency Parsing Techniques for Information Extraction Giuseppe Attardi, Maria Simi 9 Comparing State-of-the-art Dependency Parsers for the EVALITA 2014 Dependency Parsing Task Alberto Lavelli 15 Testing parsing improvements with combination and translation in Evalita 2014 Alessandro Mazzei 21 Evaluation of Events and Temporal Information EVENTI. EValuation of Events and Temporal INformation at Evalita 2014 Tommaso Caselli, Rachele Sprugnoli, Manuela Speranza and Monica Monachini 27 Experiments in Identification of Italian Temporal Expressions Giuseppe Attardi and Luca Baronti 35 HeidelTime at EVENTI: Tuning Italian Resources and Addressing TimeML’s Empty Tags Giulio Manfredi, Jannik Strötgen, Julian Zell and Michael Gertz 39 FBK-HLT-time: a complete Italian Temporal Processing system for EVENTI-Evalita 2014 Paramita Mirza and Anne-Lyse Minard 44 VI Questo e-book appartiene a AlessandroLenci a appartiene e-book Questo Sentiment Polarity Classification Overview of the Evalita 2014 SENTIment POLarity Classification Task Valerio Basile, Andrea Bolioli, Viviana Patti, Paolo Rosso and Malvina Nissim 50 UNIBA at EVALITA 2014-SENTIPOLC Task: Predicting tweet sentiment polarity combining micro-blogging, lexicon and semantic features Pierpaolo Basile and Nicole Novielli 58 ITGETARUNS A Linguistic Rule-Based System for Pragmatic Text Processing Rodolfo Delmonte 64 Subjectivity, Polarity And Irony Detection: A Multi-Layer Approach Elisabetta Fersini, Enza Messina, Federico Alberto Pozzi 70 IRADABE: Adapting English Lexicons to the Italian Sentiment Polarity Classification task Irazú Hernández Farias, Davide Buscaldi and Belém Priego Sánchez 75 Linguistically–motivated and Lexicon Features for Sentiment Analysis of Italian Tweets Andrea Cimino, Stefano Cresci, Felice Dell’Orletta and Maurizio Tesconi 81 The CoLing Lab system for Sentiment Polarity Classification of tweets Lucia C. Passaro, Gianluca E. Lebani, Laura Pollacci, Emmanuele Chersoni and Alessandro Lenci 87 The FICLIT+CS@UniBO System at the EVALITA 2014 Sentiment Polarity ClassificationTask Pierluigi Di Gennaro, Arianna Rossi and Fabio Tamburini 93 A Multiple Kernel Approach for Twitter Sentiment Analysis in Italian Giuseppe Castellucci, Danilo Croce, Diego De Cao and Roberto Basili 98 Relying on intrinsic word features to characterise subjectivity, polarity and irony of Tweets Francesco Barbieri, Francesco Ronzano and Horacio Saggion 104 Self-Evaluating Workflow for Language-Independent Sentiment Analysis Arseni Anisimovich 108 VII Questo e-book appartiene a AlessandroLenci