DOCSLIB.ORG

Temporal Models of Streaming Social Media Data

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Temporal Models of Streaming Social Media Data Temporal models of streaming social media data Daniel Preot¸iuc-Pietro Department of Computer Science The University of Sheffield A thesis submitted for the degree of Doctor of Philosophy June 2014 Abstract There are significant temporal dependencies between online behaviour and occurring real world activities. Particularly in text modelling, these are usually ignored or at best dealt with in overly simplistic ways such as assuming smooth variation with time. Social media is a new data source which present collective behaviour much more richly than tradi- tional sources, such as newswire, with a finer time granularity, timely reflection of activities, multiple modalities and large volume. Analysing temporal patterns in this data is important in order to discover newly emerging topics, periodic occurrences and correlation or causality to real world indicators or human behaviour patterns. With these opportunities come many challenges, both engineering (i.e. data volume and process- ing) and algorithmic, namely the inconsistency and short length of the messages and the presence of large amounts of irrelevant messages to our goal. Equipped with a better understanding of the dynamics of the com- plex temporal dependencies, tasks such as classification can be augmented to provide temporally aware responses. In this thesis we model the temporal dynamics of social media data. We first show that temporality is an important characteristic of this type of data. Further comparisons and correlation to real world indicators show that this data gives a timely reflection of real world events. Our goal is to use these variations to discover emerging or recurring user behaviours. We consider both the use of words and user behaviour in social media. With these goals in mind, we adapt existing and build novel machine learning techniques. These span a wide range of models: from Markov models to regularised regression models and from evolutionary spectral clustering which models smooth temporal variation to Gaussian Process regression which can identify more complex temporal patterns. We introduce approaches which discover and predict words, topics or be- haviours that change over time or occur with some regularity. These are modeled for the first time in the NLP literature by using Gaussian Processes. We demonstrate that we can effectively pick out patterns, in- cluding periodicities, and achieve state-of-the-art forecasting results. We show that this performance gain transfers to improve tasks which do not take temporal information in account. Further analysed is how temporal variation in the text can be used to discover and track new content. We develop a model that exploits the variation in word co-occurrences for clustering over time. Different collection and processing tools, as well as several datasets of social media data have been developed and published as open-source software. The thesis posits that temporal analysis of data, from social media in par- ticular, provides us with insights into real-world dynamics. Incorporating this temporal information into other applications can benefit standard tasks in natural language processing and beyond. Acknowledgements I would first like to thank my supervisor, Trevor Cohn, for guiding me through this hard but rewarding process. His expertise, vision and drive had a major impact in my career as a researcher. He has provided me patience, understanding and great feedback when I needed it most. The Department of Computer Science at the University of Sheffield and the Trendminer project offered me a great environment for research. I had the privilege to collaborate with great colleagues such as Vasileios Lampos, Sina Samangooei, Nikolaos Aletras, Andrea Varga, Kalina Bontcheva, and Dominic Rout. I would also like to thank my thesis examiners, Lucia Specia and Nello Cristianini, for their expertise and attention to detail. Without their help, my thesis would be poorer. A special thanks is also for Noah Smith and Chris Dyer which have kindly invited me to visit Carnegie Mellon University during my studies. My experience there was invaluable and has heavily and positively impacted my research. However, I would have never chosen this path without the time I spent at the University of Bucharest. Responsible for my career path was mostly Florentina Hristea who introduced me to AI and NLP in particular. The enthusiasm of my professors there, especially Marius Popescu, Liviu Dinu and Horia Georgescu, provided an inspiration for me. On a different, personal scale, none of this was possible without the help of my parents, Valeriu Preot¸iuc-Pietro and Ana-Romanit¸a Preot¸iuc-Pietro, and my girlfriend Ioana Pal. They have given me the support I needed while I was away for my studies. Contents 1 Introduction 1 1.1 Aims . .3 1.2 Thesis structure . .4 1.3 Published material . .5 2 Online Social Networks 7 2.1 Microblogging . .7 2.1.1 Conventions . .9 2.1.2 Quantitative analysis . 11 2.1.3 Data . 12 2.2 Location Based Social Networks . 15 2.2.1 Quantitative analysis . 16 2.2.2 Data . 18 2.2.3 Data analysis . 19 2.3 Conclusion . 23 3 Architecture for experimental setup 25 3.1 Components . 25 3.1.1 Tokenisation . 26 3.1.2 Language detector . 27 3.1.3 Stemmer . 28 3.1.4 Local time . 28 3.1.5 Deduplicator . 28 3.1.6 Sentiment detector . 29 3.1.7 Geolocator . 30 3.1.8 Possible extensions . 33 3.2 Open-source tool . 35 3.2.1 Aims . 35 i 3.2.2 Architecture . 36 3.2.3 Data format . 37 3.2.4 Running times . 38 3.3 Conclusions . 38 4 Temporality and social media 40 4.1 Non-temporal models . 41 4.1.1 Timestamped data analysis . 41 4.1.2 Forecasting . 44 4.2 Temporal models . 47 4.2.1 Online learning . 47 4.2.2 Online clustering and event detection . 48 4.2.3 Regularised learning . 50 4.2.4 Generative models . 51 4.3 Forecasting political voting intention . 52 4.3.1 Voting intention scores . 52 4.3.2 Linear regression . 54 4.3.3 Bilinear regression . 55 4.3.4 Experimental setup . 60 4.3.5 Quantitative results . 62 4.3.6 Qualitative analysis . 62 4.4 Non-stationary models of forecasting . 66 4.4.1 Online bilinear learning . 66 4.4.2 Non-stationarity . 71 4.5 Conclusion . 73 5 Temporal prediction 75 5.1 Gaussian Processes . 75 5.1.1 Gaussian Process regression . 76 5.1.2 Kernels . 78 5.1.3 Model selection and optimisation . 82 5.2 Temporal modelling of words . 84 5.2.1 Task . 85 5.2.2 Related work . 86 5.2.3 Experimental setup . 86 5.2.4 Methods . 87 5.2.5 Results . 88 ii 5.2.6 Discussion . 92 5.2.7 Text based prediction . 92 5.3 Temporal modelling of user behaviour . 94 5.3.1 Venue categories . 95 5.3.2 Experimental setup . 99 5.3.3 Methods . 100 5.3.4 Results . 101 5.4 Conclusion . 102 6 Detection of timely events 104 6.1 Word co-occurrence . 105 6.1.1 Pointwise mutual information . 106 6.1.2 Temporal evolution . 108 6.1.3 Applications . 109 6.2 Methods . 110 6.2.1 K-means clustering . 111 6.2.2 Spectral clustering . 112 6.2.3 Evolutionary spectral clustering . 114 6.3 Experiments . 115 6.3.1 Data . 116 6.3.2 Cluster measures . 117 6.3.3 Gardenhose experiments . 117 6.3.4 Austrian Politics experiments . 126 6.3.5 Application to summarisation . 130 6.4 Temporal experiments . 131 6.5 Conclusion . 132 7 Conclusions 134 7.1 Further research . 136 7.2 Summary . 138 A Event related clusters 139 B Aggregated clusters 141 Bibliography 144 iii Chapter 1 Introduction In an Internet where user generated content is ubiquitous, systems that are aware of the temporal evolution of data are paramount for a better understanding of real world phenomena. Thus, in this thesis, we aim to develop machine learning models for streaming user generated social media data which incorporate the temporal com- ponent in order to discover topic emergence, recurrence or correlation to underlying events. The Internet has become immense1 and plays a large part in the life of most people. This is mainly due to its widespread and proliferation of devices that can access it. Many people have shifted much of their daily personal activities to the online medium like socialising, work or information gathering. This is particularly obvious by the recent burst in usage of social media and user-generated content. Reportedly, social media usage became the single most time-consuming activity on the web as measured within the U.S., overtaking search, e-mail or gaming.2 With social media, we have seen a shift in paradigms for posting online content. Before social media, and the so-called Web 2.0 in general, the content was static; there were a few hubs of information which were updated only by those who had the authority, curating the content in the process. Now, also with the aid of the technical advances, user-generated content has become prevalent online. Thus, every person using social media can be thought as being a `journalist' [Murthy, 2011] on his daily life, exhibited by him sharing news, events, moods or images. For many, these sources have become the primary source of information for time sensitive or local events. This is mostly because people trust their peers more for giving accurate more insights, comments and a complementary view compared to traditional editorial sources. The lack of curation tends to lead to swifter responses. 1http://archive.org/ 2http://cn.nielsen.com/documents/Nielsen-Social-Media-Report_FINAL_090911.pdf 1 A notorious example illustrating the timely nature of social media (i.e.
Load more

Recommended publications
  • You and AI Conversations About AI Technologies and Their Implications for Society
    You and AI Conversations about AI technologies and their implications for society SUPPORTED BY CONVERSATIONS ABOUT AI TECHNOLOGIES AND THEIR IMPLICATIONS FOR SOCIETY DeepMind1 2 CONVERSATIONS ABOUT AI TECHNOLOGIES AND THEIR IMPLICATIONS FOR SOCIETY You and AI Conversations about AI technologies and their implications for society Artificial Intelligence (AI) is the science of making computer systems smart, and an umbrella term for a range of technologies that carry out functions that typically require intelligence in humans. AI technologies already support many everyday products and services, and the power and reach of these technologies are advancing at pace. The Royal Society is working to support an environment of careful stewardship of AI technologies, so that their benefits can be brought into being safely and rapidly, and shared across society. In support of this aim, the Society’s You and AI series brought together leading AI researchers to contribute to a public conversation about advances in AI and their implications for society. CONVERSATIONS ABOUT AI TECHNOLOGIES AND THEIR IMPLICATIONS FOR SOCIETY 3 What AI can, and cannot, do The last decade has seen exciting developments in AI – and AI researchers are tackling some fundamental challenges to develop it further AI research seeks to understand what happens or inputs do not follow a standard intelligence is, and then recreate this through pattern, these systems cannot adapt their computer systems that can automatically rules or adjust their approach. perform tasks that require some level of reasoning or intelligence in humans. In the last decade, new methods that use learning algorithms have helped create In the past, AI research has concentrated computer systems that are more flexible on creating detailed rules for how to carry and adaptive, and Demis Hassabis FRS out a task and then developing computer (co-founder, DeepMind) has been at the systems that could carry out these rules; forefront of many of these developments.
    [Show full text]
  • Annual Report of the Faculty 2018-19* *A Small Number of Items That Fall Into 2019-20 Which Are of Current Relevant Interest Are Also Included in This Report
    Annual Report of the Faculty 2018-19* *A small number of items that fall into 2019-20 which are of current relevant interest are also included in this report. INTRODUCTION This year was the first full year with Prof. Ann Copestake as Head of Department. The Department’s strategy is to continue to be a home for the world’s best computer science researchers, providing a flexible environment for working collaboratively with each other and partners of all kinds, in an open culture. Our people are part of a wider network of alumni and associates, and these connections sustain the creation of strong and revolutionary fundamental computer science research ideas, many of which will transform practice outside the academy. We continue to deliver exceptional research both individually and in dynamic teams working together across fields, connecting deep fundamentals with the wider context of research and applications in diverse fields, and we translate many of our ideas into practical innovations in whatever form is most appropriate. The University’s Strategic Research Review of the Department in September 2018 provided a valuable opportunity for reflection and strengthening how this strategy is enabled in practice. This year has seen a range of appointments which enhance our links with important fields, including climate change, and development of professional services teams to support the Department’s current size. We continue to improve our support for early career researchers and postgraduate teaching, and to communicate more effectively about our work and opportunities to collaborate with us. In a time of great excitement about data science, almost all our research groups include machine learning activities in diverse forms, and we are exploring ways to unlock potential applications across Schools and build capacity across the research base.
    [Show full text]
  • Recurrent Gaussian Processes and Robust Dynamical Modeling
    UNIVERSIDADE FEDERAL DO CEARA´ CENTRO DE TECNOLOGIA DEPARTAMENTO DE ENGENHARIA DE TELEINFORMATICA´ PROGRAMA DE POS-GRADUA¸C´ AO~ EM ENGENHARIA DE TELEINFORMATICA´ DOUTORADO EM ENGENHARIA DE TELEINFORMATICA´ CESAR´ LINCOLN CAVALCANTE MATTOS RECURRENT GAUSSIAN PROCESSES AND ROBUST DYNAMICAL MODELING FORTALEZA 2017 CESAR´ LINCOLN CAVALCANTE MATTOS RECURRENT GAUSSIAN PROCESSES AND ROBUST DYNAMICAL MODELING Tese apresentada ao Curso de Doutorado em Engenharia de Teleinform´atica do Programa de P´os-Gradua¸c~aoem Engenharia de Teleinform´atica do Centro de Tecnologia da Universidade Federal do Cear´a, como requisito parcial `a obten¸c~ao do t´ıtulo de doutor em Engenharia de Teleinform´atica. Area´ de Concentra¸c~ao: Sinais e sistemas Orientador: Prof. Dr. Guilherme de Alencar Barreto FORTALEZA 2017 Dados Internacionais de Catalogação na Publicação Universidade Federal do Ceará Biblioteca Universitária Gerada automaticamente pelo módulo Catalog, mediante os dados fornecidos pelo(a) autor(a) M39r Mattos, César Lincoln. Recurrent Gaussian Processes and Robust Dynamical Modeling / César Lincoln Mattos. – 2017. 189 f. : il. color. Tese (doutorado) – Universidade Federal do Ceará, Centro de Tecnologia, Programa de Pós-Graduação em Engenharia de Teleinformática, Fortaleza, 2017. Orientação: Prof. Dr. Guilherme de Alencar Barreto. 1. Processos Gaussianos. 2. Modelagem dinâmica. 3. Identificação de sistemas não-lineares. 4. Aprendizagem robusta. 5. Aprendizagem estocástica. I. Título. CDD 621.38 CESAR´ LINCOLN CAVALCANTE MATTOS RECURRENT GAUSSIAN PROCESSES AND ROBUST DYNAMICAL MODELING A thesis presented to the PhD course in Teleinformatics Engineering of the Graduate Program on Teleinformatics Engineering of the Center of Technology at Federal University of Cear´ain fulfillment of the the requirement for the degree of Doctor of Philosophy in Teleinformatics Engineering.
    [Show full text]
  • Signature Redacted a Uthor
    Social Inductive Biases for Reinforcement Learning by Dhaval D.K. Adjodah B.S., Physics (2011), Massachusetts Institute of Technology M.S., Technology and Policy (2011), Massachusetts Institute of Technology Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Media Arts and Sciences at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY September 2019 @ Massachusetts Institute of Technology 2019. All rights reserved. Signature redacted A uthor .................. Q .gi'gratiFMediaArts and Sciences, Sch of Architecture and Planning, ignature redacted 2, 2019 Certified by.......... KlV 'Sandy P. Pentland Toshiba Pr fessor of Media Arts and Sciences Thesis Supe risor Accepted by......... Signature redacted MASSACUET LNSTITUTE Tod Machover OF TECHNOLOGY (2 Academic Head 0 OCT 0 4 2019 Program in Media Arts and Sciences LIBRARIES ci 77 Massachusetts Avenue Cambridge, MA 02139 MITLibraries http://Iibraries.mit.edu/ask DISCLAIMER NOTICE The pagination in this thesis reflects how it was delivered to the Institute Archives and Special Collections. The Table of Contents does not accurately represent the page numbering. Social Inductive Biases for Reinforcement Learning by Dhaval D.K. Adjodah Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, on July 2, 2019, in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Media Arts and Sciences Abstract How can we build machines
    [Show full text]
  • Beneficial AI 2017
    Beneficial AI 2017 Participants & Attendees 1 Anthony Aguirre is a Professor of Physics at the University of California, Santa Cruz. He has worked on a wide variety of topics in theoretical cosmology and fundamental physics, including inflation, black holes, quantum theory, and information theory. He also has strong interest in science outreach, and has appeared in numerous science documentaries. He is a co-founder of the Future of Life Institute, the Foundational Questions Institute, and Metaculus (http://www.metaculus.com/). Sam Altman is president of Y Combinator and was the cofounder of Loopt, a location-based social networking app. He also co-founded OpenAI with Elon Musk. Sam has invested in over 1,000 companies. Dario Amodei is the co-author of the recent paper Concrete Problems in AI Safety, which outlines a pragmatic and empirical approach to making AI systems safe. Dario is currently a research scientist at OpenAI, and prior to that worked at Google and Baidu. Dario also helped to lead the project that developed Deep Speech 2, which was named one of 10 “Breakthrough Technologies of 2016” by MIT Technology Review. Dario holds a PhD in physics from Princeton University, where he was awarded the Hertz Foundation doctoral thesis prize. Amara Angelica is Research Director for Ray Kurzweil, responsible for books, charts, and special projects. Amara’s background is in aerospace engineering, in electronic warfare, electronic intelligence, human factors, and computer systems analysis areas. A co-founder and initial Academic Model/Curriculum Lead for Singularity University, she was formerly on the board of directors of the National Space Society, is a member of the Space Development Steering Committee, and is a professional member of the Institute of Electrical and Electronics Engineers (IEEE).
    [Show full text]
  • Project Final Report
    PROJECT FINAL REPORT Grant Agreement number: 305626 Project acronym: RADIANT Project title: Rapid development and distribution of statistical tools for high-throughput sequencing data Funding Scheme: Collaborative Project (FP7-HEALTH-2012-2.1.1-3: Statistical methods for collection and analysis of -omics data) Period covered: from 1.12.2013 to 30.11.2015 Name of the scientific representative of the project's co-ordinator1, Title and Organisation: Prof. Magnus Rattray. The University of Manchester Tel: 0161 275 5094 Fax: E-mail: Magnus. [email protected] Project website address: http://radiant-project.eu 1 Usually the contact person of the coordinator as specified in Art. 8.1. of the Grant Agreement. 4.1 Final publishable summary report 4.1.1 Executive summary The objectives of the RADIANT project were to develop improved computational tools for the analysis of high-throughput sequencing (HTS) data, to rapidly disseminate these tools to the wider scientific community and to support practitioners in the application of these and other advanced HTS data analysis tools in their scientific applications, lowering the bar for the widespread adoption of cutting-edge statistical tools. The project has been successful in achieving these objectives. The RADIANT project supported the development of several superb data analysis tools which have been widely adopted by practitioners. The new DESeq2 tool for differential expression analysis was developed entirely through RADIANT funding and provides substantial improvements over the earlier DESeq tool; it is one of the most widely-used tools in genomics (Love et al., 2014 cited 228 times in Scopus, 34500 software downloads in the last year).
    [Show full text]
  • Nips 2015 Conference Book Montreal 2015
    NIPS 2015 CONFERENCE BOOK MONTREAL 2015 N I P S 2 0 15 LOCATION Palais des Congrès de Montréal Convention and Exhibition Center, Montreal, Quebec, Canada TUTORIALS December 7, 2015 CONFERENCE SESSIONS December 8 - 10, 2015 SYMPOSIA Sponsored by the Neural Information Processing December 10, 2015 System Foundation, Inc The technical program includes 6 invited talks and WORKSHOPS 403 accepted papers, selected from a total of 1838 December 11-12, 2015 submissions considered by the program committee. Because the conference stresses interdisciplinary interactions, there are no parallel sessions. Papers presented at the conference will appear in “Advances in Neural Information Processing 28,” edited by edited by Daniel D. Lee, Masashi Sugiyama, Corinna Cortes, Neil Lawrence and Roman Garnett. 1 TABLE OF CONTENTS Core Logistics Team 2 Wednesday Organizing Committee 3 Oral Sessions Program Committee 3 Sessions 5 - 8, Abstracts 73 NIPS Foundation Offices and Board Members 3 Spotlights Sessions Sponsors 4 - 7 Sessions 5 - 8, Abstracts 73 Poster Sessions PROGRAM HIGHLIGHTS 8 Sessions 1 - 101 76 Exhibitors 9 Location of Presentations 80 Letter From The President 10 Abstracts 81 Future Conferences 11 Demonstrations 102 MONDAY Thursday Tutorials Oral Sessions Abstracts 13 Session 9 105 Poster Sessions Spotlights Sessions Location of Presentations 15 Session 9 105 Sessions 1 - 101 16 Poster Sessions Monday Abstracts 17 Sessions 1 - 100 105 Location of Presentations 109 Tuesday Abstracts 110 Oral Sessions Sessions 1 - 4, Abstracts 41 Symposia 131 Spotlights Sessions Workshops 132 Sessions 1 - 4, Abstracts 41 Reviewers 133 Poster Sessions Author Index 135 Sessions 1 - 101 44 Location of Presentations 48 CORE LOGISTICS TEAM Abstracts 49 The organization and management of NIPS would not be possible without the help of many volunteers, students, Demonstrations 70 researchers and administrators who donate their valuable time and energy to assist the conference in various ways.
    [Show full text]
  • Gaussian Process Modelling for Audio Signals
    School of Electronic Engineering and Computer Science Queen Mary University of London Gaussian Process Modelling for Audio Signals William J. Wilkinson PhD thesis Submitted in partial fulfilment of the requirements for the Degree of Doctor of Philosophy in Computer Science July 2019 Statement of Originality I, William J. Wilkinson, confirm that the research included within this thesis is my own work or that where it has been carried out in collaboration with, or supported by others, that this is duly acknowledged and my contribution indicated. Previously published material is also acknowledged herein. I attest that I have exercised reasonable care to ensure that the work is original, and does not to the best of my knowledge break any UK law, infringe any third party's copyright or other Intellectual Property Right, or contain any confidential material. I accept that the College has the right to use plagiarism detection software to check the electronic version of the thesis. I confirm that this thesis has not been previously submitted for the award of a degree by this or any other university. The copyright of this thesis rests with the author and no quotation from it or information derived from it may be published without the prior written consent of the author. Signature: Date: 2 Abstract Audio signals are characterised and perceived based on how their spectral make-up changes with time. Uncovering the behaviour of latent spectral components is at the heart of many real-world applications involving sound, but is a highly ill-posed task given the infinite number of ways any signal can be decomposed.
    [Show full text]
  • The Ring, No LIII, 2020-01
    The RingRingThe Hall of Fame news 1 News from the Department 2 Company spotlight: lowRISC 4 Using deep learning to improve Parkinson’s diagnoses 5 Neil Lawrence appointed DeepMind Professor 6 Cambridge Zero 7 AI for Environmental Risks 8 CHERI-ARM CPU 9 Fact-checking project wins ERC Grant 10 www.cst.cam.ac.uk/ring 2 Hall of Fame news Hall of Fame Awards The winners of last year’s awards Ring Dinner and Hall of were: We are now accepting nominations Fame Awards Ceremony for the 16th Hall of Fame awards. 1. Company of the Year: PolyAI The Cambridge Ring dinner and The awards take place annually to 2. Product of the Year: Pur3 Ltd for Hall of Fame Awards ceremony celebrate the success of companies Pixl.js will be held at Queens’ College on Thursday 2nd April 2020. founded by Computer Laboratory 3. Better Future Award: Gemma graduates and staff. Gordon for her work on bridging The winners will be announced by Category 1 – Company of the Year: virtual reality with climate change Professor Ann Copestake, and the the company must be a member of education in the LABScI Imagine awards will be presented by guest the Ring Hall of Fame. If a company project. speaker Steve Pope, Engineering Fellow at Xilinx Inc. is eligible but not currently listed, If you wish to submit a nomination please contact us to ensure that it for any of the categories, please There will be a pre-dinner lecture gets added. send nominations (explaining why from 5.15pm, given by Dr Category 2 – Product of the Year: you have nominated the company Marwa Mahmoud at the William the product concerned must have or product) to ring-organiser@cst.
    [Show full text]
  • Statistical Learning Approaches for the Control of Stormwater Systems
    Statistical Learning Approaches for the Control of Stormwater Systems by Abhiram Mullapudi A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Civil Engineering) in The University of Michigan 2020 Doctoral Committee: Associate Professor Branko Kerkez, Chair Associate Professor Andrew D. Gronewold Assistant Professor Neda Masoud Assistant Professor Ramanarayan Vasudevan Abhiram Mullapudi [email protected] ORCID iD: 0000-0001-8141-3621 © Abhiram Mullapudi 2020 DEDICATION This thesis is dedicated to my family. ii ACKNOWLEDGEMENTS I want to thank my advisor, Dr. Branko Kerkez, for his guidance and mentor- ship through my Ph.D. process. I could not have asked for a better mentor. I would also like to thank my dissertation committee: Dr. Neda Masoud, Dr. An- drew D. Gronewold, and Dr. Ramanarayan Vasudevan. Their invaluable insights helped shape my dissertation. This work has been made possible by the Na- tional Science Foundation (grant number #1737432) and Great Lakes Protection Fund (grant number #1035). I want to acknowledge all the people who helped guide my research. Dr. Matthew J. Lewis and Dr. Cyndee L. Gruden, for their help, getting me started with the Reinforcement Learning and Stormwater systems. Dr. Sara P. Rimer, for all the fascinating discussions. Dr. Jonathan L. Goodall, Dr. Jon M. Hathaway, and all the Smart and Connected Communities team, for all the fantastic conver- sations over the years. They have been instrumental in framing the questions I pursued in my dissertation. I would also like to acknowledge the graduate stu- dents in the CEE and, in particular, the members of the Real-time Water Systems Lab for the amazing five years.
    [Show full text]
  • Variational Algorithms for Approximate Bayesian Inference
    VARIATIONAL ALGORITHMS FOR APPROXIMATE BAYESIAN INFERENCE by Matthew J. Beal M.A., M.Sci., Physics, University of Cambridge, UK (1998) The Gatsby Computational Neuroscience Unit University College London 17 Queen Square London WC1N 3AR A Thesis submitted for the degree of Doctor of Philosophy of the University of London May 2003 Abstract The Bayesian framework for machine learning allows for the incorporation of prior knowledge in a coherent way, avoids overfitting problems, and provides a principled basis for selecting between alternative models. Unfortunately the computations required are usually intractable. This thesis presents a unified variational Bayesian (VB) framework which approximates these computations in models with latent variables using a lower bound on the marginal likelihood. Chapter 1 presents background material on Bayesian inference, graphical models, and propaga- tion algorithms. Chapter 2 forms the theoretical core of the thesis, generalising the expectation- maximisation (EM) algorithm for learning maximum likelihood parameters to the VB EM al- gorithm which integrates over model parameters. The algorithm is then specialised to the large family of conjugate-exponential (CE) graphical models, and several theorems are presented to pave the road for automated VB derivation procedures in both directed and undirected graphs (Bayesian and Markov networks, respectively). Chapters 3-5 derive and apply the VB EM algorithm to three commonly-used and important models: mixtures of factor analysers, linear dynamical systems, and hidden Markov models. It is shown how model selection tasks such as determining the dimensionality, cardinality, or number of variables are possible using VB approximations. Also explored are methods for combining sampling procedures with variational approximations, to estimate the tightness of VB bounds and to obtain more effective sampling algorithms.
    [Show full text]
  • AI Sector Deal One Year On
    AI Sector Deal One Year On 1 Version: T1MC00K_V1 Icons made by Freepik and Eucalyp (page 9 only) from www.flaticon.com. 2 Background 4 AI Sector Deal Achievements6 Business Environment7 Infrastructure8 The AI Council 10 Place 12 Ideas 12 People 15 Highlights 16 Looking ahead 19 AI Sector Deal One Year On 3 Background Artificial intelligence (AI) and machine Where are we today? learning are already transforming the global economy, using vast datasets to One year after the publication of theAI perform complex tasks. These range Sector Deal in April 2018 last year, from helping doctors diagnose medical government and industry have already conditions such as cancer more effectively seen significant progress. to allow people to communicate across the globe using speech recognition and The actions to date have been focusing on translation software. building the skills, talent and leadership in the UK, promoting adoption across The AI Sector Deal – a £1 billion package sectors, and ensuring AI and related of support from government and industry technologies are used safely and ethically. - is boosting the UK’s global position as a leader in developing AI and related There have been important achievements technologies. It is taking tangible actions across the Industrial Strategy’s five to advance the Industrial Strategy’s AI and foundations of productivity. Data Grand Challenge and ensure the UK is the leading destination for AI innovation and investment. The UK’s potential in AI Add£232Bn to Boost productivity Generate savings the UK economy
    [Show full text]