Human-AI Collaboration for Natural Language Generation With Interpretable Neural Networks The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Gehrmann, Sebastian. 2020. Human-AI Collaboration for Natural Language Generation With Interpretable Neural Networks. Doctoral dissertation, Harvard University, Graduate School of Arts & Sciences. Citable link https://nrs.harvard.edu/URN-3:HUL.INSTREPOS:37365160 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks a dissertation presented by Sebastian Gehrmann to The John A. Paulson School of Engineering and Applied Sciences in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the subject of Computer Science Harvard University Cambridge, Massachusetts October 2019 ©2019 – Sebastian Gehrmann Creative Commons Attribution License 4.0. You are free to share and adapt these materials for any purpose if you give appropriate credit and indicate changes. Thesis advisors: Barbara J. Grosz, Alexander M. Rush Sebastian Gehrmann Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks Abstract Using computers to generate natural language from information (NLG) requires approaches that plan the content and structure of the text and actualize it in fluent and error-free language. The typical approaches to NLG are data-driven, which means that they aim to solve the problem by learning from annotated data. Deep learning, a class of machine learning models based on neural networks, has become the standard data-driven NLG approach. While deep learning approaches lead to increased performance, they replicate undesired biases from the training data and make inexplicable mistakes. As a result, the outputs of deep learning NLG models cannot be trusted. Wethus need to develop ways in which humans can provide oversight over model outputs and retain their agency over an otherwise automated writing task. This dissertation argues that to retain agency over deep learning NLG models, we need to design them as team members instead of autonomous agents. We can achieve these team member models by considering the interaction design as an integral part of the machine learning model development. We identify two necessary conditions of team member-models – interpretability and controllability. The models need to follow a reasoning process such that human team members can understand and comprehend it. Then, if humans do not agree with the model, they should be able to change the reasoning process, and the model should adjust its output accordingly. In the first part of the dissertation, we present three case studies that demonstrate how interactive interfaces can positively affect how humans understand model predictions. In the second part,we introduce a neural network-based approach to document summarization that directly models the se- lection of relevant content. We show that, through this selection, a human user can control what part of a document the algorithm summarizes. In the final part of this dissertation, we show that thisde- sign approach, coupled with an interface that exposes these interactions, can lead to a forth and back between human and autonomous agents where the two actors collaboratively generate text. This dis- sertation thus demonstrates how to develop models with these properties and how to design neural networks as team members instead of autonomous agents. iii Contents 1 Introduction 1 1.1 Thesis Overview .................................. 7 1.2 Contributions ................................... 11 2 Natural Language Generation 13 2.1 Notation ...................................... 15 2.2 Natural Language Generation Tasks ........................ 15 2.3 Deep Learning for Natural Language Processing . 20 2.4 Approximate Search and Text Generation ..................... 34 3 Understanding Users and their Interpretability Needs 36 3.1 User Types: Architects, Trainers, and End Users . 42 3.2 Design Space for Integrating Machine Learning into Interfaces . 44 4 Evaluating Explanations in Automated Patient Phenotyping 51 4.1 Phenotyping in large EHR datasets ......................... 54 4.2 Data ........................................ 56 4.3 Methods ...................................... 57 4.4 Deriving Salient Features .............................. 60 4.5 Evaluation ..................................... 62 4.6 Results ....................................... 64 4.7 Discussion and Conclusion ............................. 68 5 Interactively Understanding Recurrent Neural Networks 70 5.1 Visualization for understanding neural networks . 72 5.2 User Analysis and Goals .............................. 73 5.3 Design of LSTMVis ................................ 75 5.4 Use Cases ...................................... 81 5.5 Long-Term Case Study ............................... 86 5.6 Conclusion ..................................... 88 iv 6 Debugging Predictions of Sequence-to-Sequence Models 90 6.1 Motivating Case Study: Debugging Translation . 94 6.2 Goals and Tasks ................................... 98 6.3 Design of Seq2Seq-Vis .............................. 101 6.4 Use Cases ...................................... 106 6.5 Conclusions .................................... 111 7 Bottom-Up Summarization: Extending Models with Controllable Variables 113 7.1 Related Work .................................... 116 7.2 Background: Neural Summarization . 118 7.3 Bottom-Up Attention ............................... 119 7.4 Inference ...................................... 123 7.5 Data and Experiments ............................... 124 7.6 Results ....................................... 125 7.7 Analysis and Discussion .............................. 128 7.8 Conclusion ..................................... 132 8 Collaborative Semantic Inference 134 8.1 Interactive Collaboration .............................. 136 8.2 Rearchitecting models to enable collaborative semantic inference . 139 8.3 Use Case: A Collaborative Summarization Model . 143 8.4 Details on the Summarization Model Hooks . 145 8.5 Towards a Co-Design Process for CSI Systems . 154 8.6 Conclusions .................................... 156 9 Discussion and Conclusion 158 9.1 The evaluation of text-generating models . 159 9.2 The ethical permissibility of text-generating models . 162 9.3 Collaboration in other domains . 166 9.4 Conclusion ..................................... 167 References 169 v Acknowledgments In graduate school, you are thrown into the deep water and your advisors teach you to swim as you go along. The first time I wrote a paper, my advisors told me that they would like me to writeitagain. For my latest paper, I received small editorial comments. I am, therefore, confident that I have learned to swim. For that, I would like to thank my advisors Barbara Grosz and Sasha Rush, who have given me the opportunity to become a researcher and have supported me in so many ways. Barbara was first person who exposed me to AI, and I am forever grateful for taking metaking on as a student and believing in me to grow as a researcher. I know most things I know about NLP thanks for Sasha’s guidance who imprinted on me his striving for flawlessly presented arguments and well-crafted code. Their support has been unwavering and their advice and subtle nudges continue to steer me in the right direction. T list of people who helped make this dissertation possible continues with Stuart Shieber, who never fails to amaze with his thoughtful suggestions and impromptu lectures on his whiteboard. I am grateful to Krzysztof Gajos for accepting me as a perpetual guest into the HCI group. Our conversa- tions indirectly influenced much of this dissertation. I will miss our 11am tea’s and participating in paper trashing sessions. Yet another person who I am happy to count as an advisor and friend is Ofra Amir. She guided my first steps in the research world, first as advisor and later as an officemate.Her advice is always useful, be it about AI or hiking. Despite my great advisors, my research would be nothing without the support from my collabo- rators. This list cannot start without thanking Hendrik Strobelt, who over the last years has been in- strumental in most work presented in this dissertation. Despite our different backgrounds, we some- how learned each others language and managed to successfully work on projects. Discussions are always better over cake. Continuing the list of amazing collaborators, I would further like to thank David Grant, Eric Carlson, Falcon Dai, Franck Dernoncourt, Hanspeter Pfister, Henry Elder, Jody Lin, Joy Wu, Lauren Urke, Lee Sanders, Leo Celi, Michael Behrisch, Mirac Suzgun, Ned Moseley, Patrick Tyler, Payel Das, Robert Krüger, Steven Layne, Tom Sercu, Yeran Li, Yonatan belinkov, Yun- tian Deng, and Zach Ziegler for all their contributions toward making this dissertation possible. I am also extremely thankful for all the great experiences I had during my internships. I got to work with a group of wonderful people at the Bruno Kessler Foundation in Italy, most of all Alessandro Cappelletti, Eleonora Mencarini, Gianluca Shiavo, Massimo Zancanaro, and Oliviero Stock. Simi- larly, I happily look at back