Extraction of Predictive Document Spans with Neural Attention
SpanPredict: Extraction of Predictive Document Spans with Neural Attention Vivek Subramanian, Matthew Engelhard, Samuel Berchuck, Liqun Chen, Ricardo Henao, and Lawrence Carin Duke University {vivek.subramanian, matthew.engelhard, samuel.berchuck, liqun.chen, ricardo.henao, lcarin}@duke.edu Abstract clinical notes, for example, attributing predictions to specific note content assures clinicians that the In many natural language processing applica- tions, identifying predictive text can be as im- model is not relying on data artifacts that are not portant as the predictions themselves. When clinically meaningful or generalizable. Moreover, predicting medical diagnoses, for example, this process may illuminate previously unknown identifying predictive content in clinical notes risk factors that are described in clinical notes but not only enhances interpretability, but also al- not captured in a structured manner. Our work is lows unknown, descriptive (i.e., text-based) motivated by the problem of autism spectrum dis- risk factors to be identified. We here formal- order (ASD) diagnosis, in which many early symp- ize this problem as predictive extraction and toms are behavioral rather than physiologic, and address it using a simple mechanism based on linear attention. Our method preserves dif- are documented in clinical notes using multiple- ferentiability, allowing scalable inference via word descriptions, not individual terms. Morever, stochastic gradient descent. Further, the model extended and nuanced descriptions are important decomposes predictions into a sum of contri- in many common document classification tasks, for butions of distinct text spans. Importantly, we instance, the scoring of movie or food reviews. require only document labels, not ground-truth Identifying important spans of text is a recurring spans.
[Show full text]