Applying Universal Schemas for Domain Specific Ontology Expansion
Total Page:16
File Type:pdf, Size:1020Kb
Applying Universal Schemas for Domain Specific Ontology Expansion Paul Groth, Sujit Pal, Darin McBeath, Brad Allen and Ron Daniel p.groth, sujit.pal, d.mcbeath, b.allen, r.daniel @elsevier.com { Elsevier Labs } 1600 John F. Kennedy Boulevard, Suite 1800, Philadelphia, PA Abstract domain experts in their ontology maintenance task. We apply these approaches to two unique resources: Manually created large scale ontologies are useful for organizing, searching, and repur- The full text of all 14 million documents (jour- • posing content ranging from scientific papers nal papers and book chapters) within Elsevier’s and medical guidelines to images. However, ScienceDirect database. This covers over 24 maintenance of such ontologies is expensive. In this paper, we investigate the use of univer- major disciplines and over 2500 journals. The Elsevier Merged Medical Taxonomy (EM- sal schemas (Riedel et al., 2013) as a mech- • anism for ontology maintenance. We ap- MeT) - a manually curated ontology contain- ply this approach on top of two unique data ing nearly one million concepts, three million sources: 14 million full-text scientific articles synonyms, more than 30 relation types, and and chapters, plus a 1 million concept hand- more than three million instances of relations curated medical ontology. We show that using between those concepts. a straightforward matrix factorization algo- rithm one can achieve 0.7 F1 measure on a link EMMeT is used within a number of Elsevier’s prediction task in this environment. Link pre- diction results can be used to suggest new rela- search engines to provide structured search results. tion types and relation type synonyms coming For example, within Clinical Key (a literature search from the literature as well as predict specific engine for clinicians), a user may search for ”breast new relation instances in the ontology. cancer” but wants to know specific treatment proce- dures for a particular sub type of the cancer (e.g. Ma- lignant Neoplasm of the breast outer quadrant). EM- 1 Introduction MeT allows for the traversal from superclass to sub- Scholarly information has been a key domain of in- class to procedure to be made. However, the main- terest for the automated knowledge base construc- tenance of EMMeT is a time consuming process tion (AKBC) community (Ororbia II et al., 2014). A requiring domain experts (e.g. trained doctors) to range of techniques have been applied to wide vari- read the literature, update, and curate the ontology. ety of tasks including: the construction of pathway This includes not only the addition of new rela- databases (Friedman et al., 2001), genomic knowl- tions but also revising existing relations, adding new edge extraction (Poon et al., 2014); scientific ques- synonyms and finding additional evidence for state- tion answering (Clark et al., 2016), and scientific ments within the ontology. We also aim to show that entity search (Sinha et al., 2015). automated approaches can provide a quick and rela- In this work, we focus on augmenting an existing tively high quality entry point that can augment cu- rich domain specific ontology with additional infor- rated knowledge bases. This work is an initial step mation. We show that existing well-known unsuper- in applying automated knowledge base techniques vised approaches can generate interesting input for in this setting. 81 Proceedings of AKBC 2016, pages 81–85, San Diego, California, June 12-17, 2016. c 2016 Association for Computational Linguistics An important aspect of this work is that it shows example of why it is helpful to employ a distributed that a rather straightforward implementation of the computing framework such as Spark. universal schema approach (Riedel et al., 2013) to Ontology Ingestion To combine the lemmatized relation extraction can be effective in a domain spe- surface form relations from the prior step with an cific setting. Prior work has used universal schemas imported ontology, we first ingest the ontology into for more general encyclopedic knowledge bases an annotation engine. We have developed the Solr such as Freebase or TAC KBP datasets. Here, we Dictionary Annotator (SoDA)1, a high performance apply it to the medical domain. Our implementation dictionary based annotation engine for Spark. SoDA is built in the Spark distributed computing frame- is specifically designed to support large dictionaries work (Zaharia et al., 2010). For a discussion and such as EMMeT. Because of the diversity of scien- comparison of universal schemas to other AKBC tific content and the availability of large ontologies, methods, we refer the reader to Section 2 of (Verga performing concept recognition type tasks using a et al., 2015), which provides an excellent overview. dictionary approach is often effective. Importantly, The contributions of this paper are as follows: this approach allows us to adjust our knowledge base to different domains by ingesting different ontolo- a confirmation that the universal schema ap- • gies. proach can be effective in a domain specific set- Concept Resolution In this stage, we run SoDA tings; and against the noun phrase arguments of the lemma- tized surface form relation instances. It matches the an exemplar of how straightforward AKBC • arguments to concepts within the ingested ontology. methods can be applied using widely available This process includes fuzzy string matching against compute platforms. all the synonyms of the various concepts. The out- put of this step is a knowledge graph consisting of We begin with a description of our system followed known concepts linked by both surface form relation by an set of initial experiments and then conclude. instances as well as by relations from the ontology. 2 System Description The concept resolution step reduces the 475 mil- lion Open IE facts to 46 million where both surface Our CAT3-KB system consists of seven steps de- form arguments match EMMeT concepts. We add picted in Figure 1: open information extraction, on- in the three million facts from EMMeT, giving us a tology ingestion, concept resolution, matrix con- medical knowledge base of 49 million facts. struction, matrix factorization, matrix completion, Matrix Construction Following (Riedel et al., and curation. We now describe these steps in turn. 2013), we construct a matrix from those 49 million Open IE ScienceDirect content consists of XML facts. The rows are the pairs of arguments in each representations of articles. From this XML, we ex- fact, and the columns are the lemmatized relations tract the plain text of articles. These articles are (or the known relations from EMMeT). The cells fed through a Spark-based reimplementation of Re- have binary values; one where the two arguments Verb (Fader et al., 2011). At a high level, ReVerb are linked by that relation, zero where not. As de- identifies relation phrases by looking for text spans scribed later, our initial experiments use subsets of starting with a verb and ending with a preposition this matrix. (e.g. ”is the leading cause of”). Noun phrases be- Matrix Factorization (Riedel et al., 2013) fore and after the span are used as the arguments presents a number of models based on the universal of the subsequent relation instance (i.e. fact). We schema representation. A key insight of that work lemmatize all relations, and only keep relation types was that these models could take advantage of tech- that have more than 5 distinct argument pairs and niques from collaborative filtering. We apply this that occur more than 25 times. These parameters insight directly and build a model using alternating are adjustable. From the original 14 million articles 1 https://databricks.com/blog/2016/02/10/how-elsevier- (representing approximately 1 TB of text), 475 mil- labs-implemented-dictionary-annotation-at-scale-with-apache- lion facts are returned. This amount of data is an spark-on-databricks.html 82 Open Curation Content Information Surface form Extraction relations ConceptEntity Knowledge Matrix Universal Matrix Factorization Matrix Predicted ResolutionResolution Construction Factorization Completion graph schema model relations Triple Structured Taxonomy Extraction relations Figure 1: The Architecture of CAT3-KB least squares (ALS) to approximate the given uni- An important point to make is that we are predict- versal schema as the product of two lower rank ma- ing new relations that do no explicitly appear in the trices. Each relation is approximated by one factor literature or in the input ontology. To clarify, while and each concept-concept pair is approximated by just looking at the Open IE facts or input knowledge another. In our setting, we use Spark’s existing par- graph relations is of interest, our goal here is to pre- allel implementation of an implicit feedback version dict new links not within that knowledge graph. of ALS with regularization. The number of itera- Curation The final step is to provide the set of tions, regularization, and implicit feedback baseline predicted facts to experts for analysis and use in confidence parameters are all set empirically. Those their application. These may be used to refine the settings are given later in the experimental section. input taxonomy through the addition of relations or From our initial observations, results appear to be inserted back into the input knowledge graph. robust to differing parameter settings but we have yet to perform a full sensitivity analysis. The output 3 Initial Experiments of this step are an Nxk and an Mxk factor matrices, We have conducted an initial set of tests with small where N is the number of concept-concept pairs and subsets of the input matrix so that we can eas- M is the number of lemmatized and EMMeT rela- ily examine the outputs manually and quickly iter- tions. The results described here are for k = 30. ate. Two subsets were created, based on concepts Matrix Completion To generate a set of pre- that appear frequently within in the Clinical Key dicted relation instances, we approximate the orig- search logs.