DOCSLIB.ORG

Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks Human-AI Collaboration for Natural Language Generation With Interpretable Neural Networks The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Gehrmann, Sebastian. 2020. Human-AI Collaboration for Natural Language Generation With Interpretable Neural Networks. Doctoral dissertation, Harvard University, Graduate School of Arts & Sciences. Citable link https://nrs.harvard.edu/URN-3:HUL.INSTREPOS:37365160 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks a dissertation presented by Sebastian Gehrmann to The John A. Paulson School of Engineering and Applied Sciences in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the subject of Computer Science Harvard University Cambridge, Massachusetts October 2019 ©2019 – Sebastian Gehrmann Creative Commons Attribution License 4.0. You are free to share and adapt these materials for any purpose if you give appropriate credit and indicate changes. Thesis advisors: Barbara J. Grosz, Alexander M. Rush Sebastian Gehrmann Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks Abstract Using computers to generate natural language from information (NLG) requires approaches that plan the content and structure of the text and actualize it in fluent and error-free language. The typical approaches to NLG are data-driven, which means that they aim to solve the problem by learning from annotated data. Deep learning, a class of machine learning models based on neural networks, has become the standard data-driven NLG approach. While deep learning approaches lead to increased performance, they replicate undesired biases from the training data and make inexplicable mistakes. As a result, the outputs of deep learning NLG models cannot be trusted. Wethus need to develop ways in which humans can provide oversight over model outputs and retain their agency over an otherwise automated writing task. This dissertation argues that to retain agency over deep learning NLG models, we need to design them as team members instead of autonomous agents. We can achieve these team member models by considering the interaction design as an integral part of the machine learning model development. We identify two necessary conditions of team member-models – interpretability and controllability. The models need to follow a reasoning process such that human team members can understand and comprehend it. Then, if humans do not agree with the model, they should be able to change the reasoning process, and the model should adjust its output accordingly. In the first part of the dissertation, we present three case studies that demonstrate how interactive interfaces can positively affect how humans understand model predictions. In the second part,we introduce a neural network-based approach to document summarization that directly models the se- lection of relevant content. We show that, through this selection, a human user can control what part of a document the algorithm summarizes. In the final part of this dissertation, we show that thisde- sign approach, coupled with an interface that exposes these interactions, can lead to a forth and back between human and autonomous agents where the two actors collaboratively generate text. This dis- sertation thus demonstrates how to develop models with these properties and how to design neural networks as team members instead of autonomous agents. iii Contents 1 Introduction 1 1.1 Thesis Overview .................................. 7 1.2 Contributions ................................... 11 2 Natural Language Generation 13 2.1 Notation ...................................... 15 2.2 Natural Language Generation Tasks ........................ 15 2.3 Deep Learning for Natural Language Processing . 20 2.4 Approximate Search and Text Generation ..................... 34 3 Understanding Users and their Interpretability Needs 36 3.1 User Types: Architects, Trainers, and End Users . 42 3.2 Design Space for Integrating Machine Learning into Interfaces . 44 4 Evaluating Explanations in Automated Patient Phenotyping 51 4.1 Phenotyping in large EHR datasets ......................... 54 4.2 Data ........................................ 56 4.3 Methods ...................................... 57 4.4 Deriving Salient Features .............................. 60 4.5 Evaluation ..................................... 62 4.6 Results ....................................... 64 4.7 Discussion and Conclusion ............................. 68 5 Interactively Understanding Recurrent Neural Networks 70 5.1 Visualization for understanding neural networks . 72 5.2 User Analysis and Goals .............................. 73 5.3 Design of LSTMVis ................................ 75 5.4 Use Cases ...................................... 81 5.5 Long-Term Case Study ............................... 86 5.6 Conclusion ..................................... 88 iv 6 Debugging Predictions of Sequence-to-Sequence Models 90 6.1 Motivating Case Study: Debugging Translation . 94 6.2 Goals and Tasks ................................... 98 6.3 Design of Seq2Seq-Vis .............................. 101 6.4 Use Cases ...................................... 106 6.5 Conclusions .................................... 111 7 Bottom-Up Summarization: Extending Models with Controllable Variables 113 7.1 Related Work .................................... 116 7.2 Background: Neural Summarization . 118 7.3 Bottom-Up Attention ............................... 119 7.4 Inference ...................................... 123 7.5 Data and Experiments ............................... 124 7.6 Results ....................................... 125 7.7 Analysis and Discussion .............................. 128 7.8 Conclusion ..................................... 132 8 Collaborative Semantic Inference 134 8.1 Interactive Collaboration .............................. 136 8.2 Rearchitecting models to enable collaborative semantic inference . 139 8.3 Use Case: A Collaborative Summarization Model . 143 8.4 Details on the Summarization Model Hooks . 145 8.5 Towards a Co-Design Process for CSI Systems . 154 8.6 Conclusions .................................... 156 9 Discussion and Conclusion 158 9.1 The evaluation of text-generating models . 159 9.2 The ethical permissibility of text-generating models . 162 9.3 Collaboration in other domains . 166 9.4 Conclusion ..................................... 167 References 169 v Acknowledgments In graduate school, you are thrown into the deep water and your advisors teach you to swim as you go along. The first time I wrote a paper, my advisors told me that they would like me to writeitagain. For my latest paper, I received small editorial comments. I am, therefore, confident that I have learned to swim. For that, I would like to thank my advisors Barbara Grosz and Sasha Rush, who have given me the opportunity to become a researcher and have supported me in so many ways. Barbara was first person who exposed me to AI, and I am forever grateful for taking metaking on as a student and believing in me to grow as a researcher. I know most things I know about NLP thanks for Sasha’s guidance who imprinted on me his striving for flawlessly presented arguments and well-crafted code. Their support has been unwavering and their advice and subtle nudges continue to steer me in the right direction. T list of people who helped make this dissertation possible continues with Stuart Shieber, who never fails to amaze with his thoughtful suggestions and impromptu lectures on his whiteboard. I am grateful to Krzysztof Gajos for accepting me as a perpetual guest into the HCI group. Our conversa- tions indirectly influenced much of this dissertation. I will miss our 11am tea’s and participating in paper trashing sessions. Yet another person who I am happy to count as an advisor and friend is Ofra Amir. She guided my first steps in the research world, first as advisor and later as an officemate.Her advice is always useful, be it about AI or hiking. Despite my great advisors, my research would be nothing without the support from my collabo- rators. This list cannot start without thanking Hendrik Strobelt, who over the last years has been in- strumental in most work presented in this dissertation. Despite our different backgrounds, we some- how learned each others language and managed to successfully work on projects. Discussions are always better over cake. Continuing the list of amazing collaborators, I would further like to thank David Grant, Eric Carlson, Falcon Dai, Franck Dernoncourt, Hanspeter Pfister, Henry Elder, Jody Lin, Joy Wu, Lauren Urke, Lee Sanders, Leo Celi, Michael Behrisch, Mirac Suzgun, Ned Moseley, Patrick Tyler, Payel Das, Robert Krüger, Steven Layne, Tom Sercu, Yeran Li, Yonatan belinkov, Yun- tian Deng, and Zach Ziegler for all their contributions toward making this dissertation possible. I am also extremely thankful for all the great experiences I had during my internships. I got to work with a group of wonderful people at the Bruno Kessler Foundation in Italy, most of all Alessandro Cappelletti, Eleonora Mencarini, Gianluca Shiavo, Massimo Zancanaro, and Oliviero Stock. Simi- larly, I happily look at back
Load more

Recommended publications
  • Ways of Seeing Data: a Survey of Fields of Visualization
    • Ways of seeing data: • A survey of fields of visualization • Gordon Kindlmann • [email protected] Part of the talk series “Show and Tell: Visualizing the Life of the Mind” • Nov 19, 2012 http://rcc.uchicago.edu/news/show_and_tell_abstracts.html 2012 Presidential Election REPUBLICAN DEMOCRAT http://www.npr.org/blogs/itsallpolitics/2012/11/01/163632378/a-campaign-map-morphed-by-money 2012 Presidential Election http://gizmodo.com/5960290/this-is-the-real-political-map-of-america-hint-we-are-not-that-divided 2012 Presidential Election http://www.npr.org/blogs/itsallpolitics/2012/11/01/163632378/a-campaign-map-morphed-by-money 2012 Presidential Election http://www.npr.org/blogs/itsallpolitics/2012/11/01/163632378/a-campaign-map-morphed-by-money 2012 Presidential Election http://www.npr.org/blogs/itsallpolitics/2012/11/01/163632378/a-campaign-map-morphed-by-money Clarifying distortions Tube map from 1908 http://www.20thcenturylondon.org.uk/beck-henry-harry http://briankerr.wordpress.com/2009/06/08/connections/ http://en.wikipedia.org/wiki/Harry_Beck Clarifying distortions http://www.20thcenturylondon.org.uk/beck-henry-harry Harry Beck 1933 http://briankerr.wordpress.com/2009/06/08/connections/ http://en.wikipedia.org/wiki/Harry_Beck Clarifying distortions Clarifying distortions Joachim Böttger, Ulrik Brandes, Oliver Deussen, Hendrik Ziezold, “Map Warping for the Annotation of Metro Maps” IEEE Computer Graphics and Applications, 28(5):56-65, 2008 Maps reflect conventions, choices, and priorities “A single map is but one of an indefinitely large
    [Show full text]
  • Automatic Correction of Real-Word Errors in Spanish Clinical Texts
    sensors Article Automatic Correction of Real-Word Errors in Spanish Clinical Texts Daniel Bravo-Candel 1,Jésica López-Hernández 1, José Antonio García-Díaz 1 , Fernando Molina-Molina 2 and Francisco García-Sánchez 1,* 1 Department of Informatics and Systems, Faculty of Computer Science, Campus de Espinardo, University of Murcia, 30100 Murcia, Spain; [email protected] (D.B.-C.); [email protected] (J.L.-H.); [email protected] (J.A.G.-D.) 2 VÓCALI Sistemas Inteligentes S.L., 30100 Murcia, Spain; [email protected] * Correspondence: [email protected]; Tel.: +34-86888-8107 Abstract: Real-word errors are characterized by being actual terms in the dictionary. By providing context, real-word errors are detected. Traditional methods to detect and correct such errors are mostly based on counting the frequency of short word sequences in a corpus. Then, the probability of a word being a real-word error is computed. On the other hand, state-of-the-art approaches make use of deep learning models to learn context by extracting semantic features from text. In this work, a deep learning model were implemented for correcting real-word errors in clinical text. Specifically, a Seq2seq Neural Machine Translation Model mapped erroneous sentences to correct them. For that, different types of error were generated in correct sentences by using rules. Different Seq2seq models were trained and evaluated on two corpora: the Wikicorpus and a collection of three clinical datasets. The medicine corpus was much smaller than the Wikicorpus due to privacy issues when dealing Citation: Bravo-Candel, D.; López-Hernández, J.; García-Díaz, with patient information.
    [Show full text]
  • When Diving Deep Into Operationalizing the Ethical Development of Artificial Intelligence, One Immediately Runs Into the “Fractal Problem”
    When diving deep into operationalizing the ethical development of artificial intelligence, one immediately runs into the “fractal problem”. This may also be called the “infinite onion” problem.1 That is, each problem within development that you pinpoint expands into a vast universe of new complex problems. It can be hard to make any measurable progress as you run in circles among different competing issues, but one of the paths forward is to pause at a specific point and detail what you see there. Here I list some of the complex points that I see at play in the firing of Dr. Timnit Gebru, and why it will remain forever after a really, really, really terrible decision. The Punchline. The firing of Dr. Timnit Gebru is not okay, and the way it was done is not okay. It appears to stem from the same lack of foresight that is at the core of modern technology,2 and so itself serves as an example of the problem. The firing seems to have been fueled by the same underpinnings of racism and sexism that our AI systems, when in the wrong hands, tend to soak up. How Dr. Gebru was fired is not okay, what was said about it is not okay, and the environment leading up to it was -- and is -- not okay. Every moment where Jeff Dean and Megan Kacholia do not take responsibility for their actions is another moment where the company as a whole stands by silently as if to intentionally send the horrifying message that Dr. Gebru deserves to be treated this way.
    [Show full text]
  • Bias and Fairness in NLP
    Bias and Fairness in NLP Margaret Mitchell Kai-Wei Chang Vicente Ordóñez Román Google Brain UCLA University of Virginia Vinodkumar Prabhakaran Google Brain Tutorial Outline ● Part 1: Cognitive Biases / Data Biases / Bias laundering ● Part 2: Bias in NLP and Mitigation Approaches ● Part 3: Building Fair and Robust Representations for Vision and Language ● Part 4: Conclusion and Discussion “Bias Laundering” Cognitive Biases, Data Biases, and ML Vinodkumar Prabhakaran Margaret Mitchell Google Brain Google Brain Andrew Emily Simone Parker Lucy Ben Elena Deb Timnit Gebru Zaldivar Denton Wu Barnes Vasserman Hutchinson Spitzer Raji Adrian Brian Dirk Josh Alex Blake Hee Jung Hartwig Blaise Benton Zhang Hovy Lovejoy Beutel Lemoine Ryu Adam Agüera y Arcas What’s in this tutorial ● Motivation for Fairness research in NLP ● How and why NLP models may be unfair ● Various types of NLP fairness issues and mitigation approaches ● What can/should we do? What’s NOT in this tutorial ● Definitive answers to fairness/ethical questions ● Prescriptive solutions to fix ML/NLP (un)fairness What do you see? What do you see? ● Bananas What do you see? ● Bananas ● Stickers What do you see? ● Bananas ● Stickers ● Dole Bananas What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas at a store What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas at a store ● Bananas on shelves What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas at a store ● Bananas on shelves ● Bunches of bananas What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas
    [Show full text]
  • Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification∗
    Proceedings of Machine Learning Research 81:1{15, 2018 Conference on Fairness, Accountability, and Transparency Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification∗ Joy Buolamwini [email protected] MIT Media Lab 75 Amherst St. Cambridge, MA 02139 Timnit Gebru [email protected] Microsoft Research 641 Avenue of the Americas, New York, NY 10011 Editors: Sorelle A. Friedler and Christo Wilson Abstract who is hired, fired, granted a loan, or how long Recent studies demonstrate that machine an individual spends in prison, decisions that learning algorithms can discriminate based have traditionally been performed by humans are on classes like race and gender. In this rapidly made by algorithms (O'Neil, 2017; Citron work, we present an approach to evaluate and Pasquale, 2014). Even AI-based technologies bias present in automated facial analysis al- that are not specifically trained to perform high- gorithms and datasets with respect to phe- stakes tasks (such as determining how long some- notypic subgroups. Using the dermatolo- one spends in prison) can be used in a pipeline gist approved Fitzpatrick Skin Type clas- that performs such tasks. For example, while sification system, we characterize the gen- face recognition software by itself should not be der and skin type distribution of two facial analysis benchmarks, IJB-A and Adience. trained to determine the fate of an individual in We find that these datasets are overwhelm- the criminal justice system, it is very likely that ingly composed of lighter-skinned subjects such software is used to identify suspects. Thus, (79:6% for IJB-A and 86:2% for Adience) an error in the output of a face recognition algo- and introduce a new facial analysis dataset rithm used as input for other tasks can have se- which is balanced by gender and skin type.
    [Show full text]
  • Point-Based Computer Graphics
    Point-Based Computer Graphics Eurographics 2002 Tutorial T6 Organizers Markus Gross ETH Zürich Hanspeter Pfister MERL, Cambridge Presenters Marc Alexa TU Darmstadt Markus Gross ETH Zürich Mark Pauly ETH Zürich Hanspeter Pfister MERL, Cambridge Marc Stamminger Bauhaus-Universität Weimar Matthias Zwicker ETH Zürich Contents Tutorial Schedule................................................................................................2 Presenters Biographies........................................................................................3 Presenters Contact Information ..........................................................................4 References...........................................................................................................5 Project Pages.......................................................................................................6 Tutorial Schedule 8:30-8:45 Introduction (M. Gross) 8:45-9:45 Point Rendering (M. Zwicker) 9:45-10:00 Acquisition of Point-Sampled Geometry and Appearance I (H. Pfister) 10:00-10:30 Coffee Break 10:30-11:15 Acquisition of Point-Sampled Geometry and Appearance II (H. Pfister) 11:15-12:00 Dynamic Point Sampling (M. Stamminger) 12:00-14:00 Lunch 14:00-15:00 Point-Based Surface Representations (M. Alexa) 15:00-15:30 Spectral Processing of Point-Sampled Geometry (M. Gross) 15:30-16:00 Coffee Break 16:00-16:30 Efficient Simplification of Point-Sampled Geometry (M. Pauly) 16:30-17:15 Pointshop3D: An Interactive System for Point-Based Surface Editing (M. Pauly) 17:15-17:30 Discussion (all) 2 Presenters Biographies Dr. Markus Gross is a professor of computer science and the director of the computer graphics laboratory of the Swiss Federal Institute of Technology (ETH) in Zürich. He received a degree in electrical and computer engineering and a Ph.D. on computer graphics and image analysis, both from the University of Saarbrucken, Germany. From 1990 to 1994 Dr. Gross was with the Computer Graphics Center in Darmstadt, where he established and directed the Visual Computing Group.
    [Show full text]
  • Unified Language Model Pre-Training for Natural
    Unified Language Model Pre-training for Natural Language Understanding and Generation Li Dong∗ Nan Yang∗ Wenhui Wang∗ Furu Wei∗† Xiaodong Liu Yu Wang Jianfeng Gao Ming Zhou Hsiao-Wuen Hon Microsoft Research {lidong1,nanya,wenwan,fuwei}@microsoft.com {xiaodl,yuwan,jfgao,mingzhou,hon}@microsoft.com Abstract This paper presents a new UNIfied pre-trained Language Model (UNILM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained using three types of language modeling tasks: unidirec- tional, bidirectional, and sequence-to-sequence prediction. The unified modeling is achieved by employing a shared Transformer network and utilizing specific self-attention masks to control what context the prediction conditions on. UNILM compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0 and CoQA question answering tasks. Moreover, UNILM achieves new state-of- the-art results on five natural language generation datasets, including improving the CNN/DailyMail abstractive summarization ROUGE-L to 40.51 (2.04 absolute improvement), the Gigaword abstractive summarization ROUGE-L to 35.75 (0.86 absolute improvement), the CoQA generative question answering F1 score to 82.5 (37.1 absolute improvement), the SQuAD question generation BLEU-4 to 22.12 (3.75 absolute improvement), and the DSTC7 document-grounded dialog response generation NIST-4 to 2.67 (human performance is 2.65). The code and pre-trained models are available at https://github.com/microsoft/unilm. 1 Introduction Language model (LM) pre-training has substantially advanced the state of the art across a variety of natural language processing tasks [8, 29, 19, 31, 9, 1].
    [Show full text]
  • Mike Roberts Curriculum Vitae
    Mike Roberts +1 (650) 924–7168 [email protected] mikeroberts3000.github.io Research Using photorealistic synthetic data for computer vision; motion planning, trajectory optimization, Interests and control methods for robotics; reconstructing 3D scenes from images; continuous and discrete optimization; submodular optimization; software tools and algorithms for creativity support Education Stanford University Stanford, California Ph.D. Computer Science 2012–2019 Advisor: Pat Hanrahan Dissertation: Trajectory Optimization Methods for Drone Cameras Harvard University Cambridge, Massachusetts Visiting Research Fellow Summer 2013 Advisor: Hanspeter Pfister University of Calgary Calgary, Canada M.S. Computer Science 2010 University of Calgary Calgary, Canada B.S. Computer Science 2007 Employment Intel Labs Seattle, Washington Research Scientist 2021– Mentor: Vladlen Koltun Apple Seattle, Washington Research Scientist 2018–2021 Microsoft Research Redmond, Washington Research Intern Summer 2016, 2017 Mentors: Neel Joshi, Sudipta Sinha Skydio Redwood City, California Research Intern Spring 2016 Mentors: Adam Bry, Frank Dellaert Udacity Mountain View, California Course Developer, Introduction to Parallel Computing 2012–2013 Instructors: John Owens, David Luebke Harvard University Cambridge, Massachusetts Research Fellow 2010–2012 Advisor: Hanspeter Pfister NVIDIA Austin, Texas Developer Tools Programmer Intern Summer 2009 Radical Entertainment Vancouver, Canada Graphics Programmer Intern 2005–2006 Honors And Featured in the Highlights of
    [Show full text]
  • Mike Roberts Curriculum Vitae
    Mike Roberts Union Street, Suite , Seattle, Washington, + () / [email protected] http://mikeroberts.github.io Research Using photorealistic synthetic data for computer vision; motion planning, trajectory optimization, Interests and control methods for robotics; reconstructing D scenes from images; continuous and discrete optimization; submodular optimization; software tools and algorithms for creativity support. Education Stanford University Stanford, California Ph.D. Computer Science – Advisor: Pat Hanrahan Dissertation: Trajectory Optimization Methods for Drone Cameras Harvard University Cambridge, Massachusetts Visiting Research Fellow Summer John A. Paulson School of Engineering and Applied Sciences Advisor: Hanspeter Pfister University of Calgary Calgary, Canada M.S. Computer Science University of Calgary Calgary, Canada B.S. Computer Science Employment Apple Seattle, Washington Research Scientist – Microsoft Research Redmond, Washington Research Intern Summer , Advisors: Neel Joshi, Sudipta Sinha Skydio Redwood City, California Research Intern Spring Mentors: Adam Bry, Frank Dellaert Udacity Mountain View, California Course Developer, Introduction to Parallel Computing – Instructors: John Owens, David Luebke Harvard University Cambridge, Massachusetts Research Fellow, John A. Paulson School of Engineering and Applied Sciences – Advisor: Hanspeter Pfister NVIDIA Austin, Texas Developer Tools Programmer Intern Summer Radical Entertainment Vancouver, Canada Graphics Programmer Intern – Honors And Featured in the Highlights
    [Show full text]
  • Monocular Neural Image Based Rendering with Continuous View Control
    Monocular Neural Image Based Rendering with Continuous View Control Xu Chen∗ Jie Song∗ Otmar Hilliges AIT Lab, ETH Zurich {xuchen, jsong, otmar.hilliges}@inf.ethz.ch ... ... Figure 1: Interactive novel view synthesis: Given a single source view our approach can generate a continuous sequence of geometrically accurate novel views under fine-grained control. Top: Given a single street-view like input, a user may specify a continuous camera trajectory and our system generates the corresponding views in real-time. Bottom: An unseen hi-res internet image is used to synthesize novel views, while the camera is controlled interactively. Please refer to our project homepagey. Abstract like to view products interactively in 3D rather than from We propose a method to produce a continuous stream discrete view angles. Likewise in map applications it is of novel views under fine-grained (e.g., 1◦ step-size) cam- desirable to explore the vicinity of street-view like images era control at interactive rates. A novel learning pipeline beyond the position at which the photograph was taken. This determines the output pixels directly from the source color. is often not possible because either only 2D imagery exists, Injecting geometric transformations, including perspective or because storing and rendering of full 3D information does projection, 3D rotation and translation into the network not scale. To overcome this limitation we study the problem forces implicit reasoning about the underlying geometry. of interactive view synthesis with 6-DoF view control, taking The latent 3D geometry representation is compact and mean- only a single image as input. We propose a method that ingful under 3D transformation, being able to produce geo- can produce a continuous stream of novel views under fine- grained (e.g., 1◦ step-size) camera control (see Fig.1).
    [Show full text]
  • Thouis R. Jones [email protected]
    Thouis R. Jones [email protected] Summary Computational biologist with 20+ years' experience, including software engineering, algorithm development, image analysis, and solving challenging biomedical research problems. Research Experience Connectomics and Dense Reconstruction of Neural Connectivity Harvard University School of Engineering and Applied Sciences (2012-Present) My research is on software for automatic dense reconstruction of neural connectivity. Using using high-resolution serial sectioning and electron microscopy, biologists are able to image nanometer-scale details of neural structure. This data is extremely large (1 petabyte / cubic mm). The tools I develop in collaboration with the members of the Connectomics project allow biologists to analyze and explore this data. Image analysis, high-throughput screening, & large-scale data analysis Curie Institute, Department of Translational Research (2010 - 2012) Broad Institute of MIT and Harvard, Imaging Platform (2007 - 2010) My research focused on image and data analysis for high-content screening (HCS) and large-scale machine learning applied to HCS data. My work emphasized making powerful analysis methods available to the biological community in user-friendly tools. As part of this effort, I cofounded the open-source CellProfiler project with Dr. Anne Carpenter (director of the Imaging Platform at the Broad Institute). PhD Research: High-throughput microscopy and RNAi to reveal gene function. My doctoral research focused on methods for determining gene function using high- throughput microscopy, living cell microarrays, and RNA interference. To accomplish this goal, my collaborators and I designed and released the first open-source high- throughput cell image analysis software, CellProfiler (www.cellprofiler.org). We also developed advanced data mining methods for high-content screens.
    [Show full text]
  • Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks
    Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks a dissertation presented by Sebastian Gehrmann to The John A. Paulson School of Engineering and Applied Sciences in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the subject of Computer Science Harvard University Cambridge, Massachusetts May 2020 ©2019 – Sebastian Gehrmann Creative Commons Attribution License 4.0. You are free to share and adapt these materials for any purpose if you give appropriate credit and indicate changes. Thesis advisors: Barbara J. Grosz, Alexander M. Rush Sebastian Gehrmann Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks Abstract Using computers to generate natural language from information (NLG) requires approaches that plan the content and structure of the text and actualize it in fuent and error-free language. The typical approaches to NLG are data-driven, which means that they aim to solve the problem by learning from annotated data. Deep learning, a class of machine learning models based on neural networks, has become the standard data-driven NLG approach. While deep learning approaches lead to increased performance, they replicate undesired biases from the training data and make inexplicable mistakes. As a result, the outputs of deep learning NLG models cannot be trusted. Wethus need to develop ways in which humans can provide oversight over model outputs and retain their agency over an otherwise automated writing task. This dissertation argues that to retain agency over deep learning NLG models, we need to design them as team members instead of autonomous agents. We can achieve these team member models by considering the interaction design as an integral part of the machine learning model development.
    [Show full text]