Open Domain Information Extraction with Harmonic Energy Functions Chirag Nagpal1, Kyle Miller and Artur Dubrawski2 1Language Technologies Institute, 2Auton Lab, Robotics Institute, Carnegie Mellon University { chiragn, mille856, awd} @ andrew.cmu.edu

Objectives Architecture Information Extraction Scalability We build upon the Harmonic Energy Functions used previously in an Active Learning Framework • Energy Minimisation is extended towards a more • Named Entities extracted with Distant Supervision Generic Open Domain Information Extraction System from external Knowledge Bases are exploited to • A structured dataset curated from Wikipedia articles of describe a Graph with weighted edges. countries is utilised to answer queries such as ’Arabic • Entities to provide context, and constrain search to Speaking African Nations’, ’Most Democratic Nations’. utilise Active Search for Information Extraction from Open Domain data. • Scalabilty issues arising due to large number of nodes are addressed with numerical techniques, that speed Figure 2: Data Preprocessing Pipeline solution of Linear Systems. To extract entities, we exploit external KBs: Figure 8: Scalability Improvements with Alegebric Multigrid Introduction • Wordnet: Named Entity Recogniser and Part of Arabic Speaking African Nations Most Democratic Nations Speech Tagger trained on the Penn Treebank Corpus Figure 5: Graph extracted from WordNet on Wikipedia pages of is employed along with Lesk, to perform Word Sense 250 countries. We compare recall by labelling certain ’interesting’ f = (D˜ + λ(D − A))−1Dy˜ 0 entities as Positive. Disambiguation.   DL 0 • Robustness is tested on the more unstructured where, D˜ =   • DBPedia: ’Spotlight’[2] an open-source tool to map 0 λw D dataset, 20 Newsgroups, to answer simple queries like 0 U text to Entities. As, E is convex in f, we can solve the given Lin- the Open Source or Gaming Operatin Systems, Sports ear System, using Algebric Multigrid[3], which is further Teams from New York, etc. speeded, by precomputing the Restriction and Prolonga- tion matrix, and reusing them in each iteration. WordNet Algorithm 1: Search with Precomputed Grids Input : w0, η, π Initialise Graph, G with the targets 1 t ← 0; Open Source Gaming 2 if t == 0 then DBPedia Figure 6: Activations of Entitites representing ’Operating Systems’, 3 A, y ← setData(G); Figure 3: Comparative Entities Extracted with WordNet and on a Graph extracted from DBPedia on 20 Newsgroups. Figure 1: In each iteration, the a label associated with the node is 4 R,P ← ComputeGrids(A); DBPedia on a representative sample. (t+1) queried. With each new label, the activations change, so as to 5 f ←solveAMG(R, P, A, y); minimise the Energy over the Graph. • Active Search, is applied to 20 Newsgroups for a 6 else One-Vs-Rest Classification task. 7 A, y ← setData(G); (t+1) (t) X 2 • AS on Bi-Partite graphs with Entities significantly f ←solveAMG(R, P, A, y, f ); arg min E(f) = (yi − fi) Dii+ 8 end f i∈L outperforms AS on Graphs with Cosine Distance X 2 X 2 λ(w0 (fi − π) Dii + (fi − fj) Aij) between nodes. References i∈U i,j New York New York + Baseball P where, Aij = K(xi, xj),Dii = j K(xi, xj) and Figure 7: Activations of Entities Representing ’Sports Teams’ on a [1] Xuezhi Wang, Roman Garnett, and Jeff Schneider. Active search on graphs. λ, w0 are regularising constants Graph extracted from DBPedia on 20 Newsgroups. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 731–738. ACM, 2013. [2] Joachim Daiber, Max Jakob, Chris Hokamp, and Pablo N. Mendes. • Active Search [1], minimises the Energy, E(f). Future Work Improving efficiency and accuracy in multilingual entity extraction. In Proceedings of the 9th International Conference on Semantic Systems (I-Semantics), 2013. • f is the ’activations’ of each node in the Graph. (t+1) (t) [3] W. N. Bell, L. N. Olson, and J. B. Schroder. • Instead of a Kernel, K(·, ·) as edge, entities are • Treat change in f w.r.t. f as Impact Factor (IF). PyAMG: Algebraic multigrid solvers in Python v3.0, 2015. Release 3.0. exploited as nodes with documents. This results in • Faster computation given IF is a function of f or f |.f bi-partite graphs over which energy minimisation is • Investigate use of a classifier, to learn IF. performed.

Figure 4: Harmonic Energy Minimisation - Active Search