3 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 2 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 3 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 4 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 5 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 6 . Types of questions
. Web search and conversational assistants
. Google, Cortana, Alexa, Siri
. What changes across languages?
. What changes across persons?
. Domains? Complex questions?
. From text? KG?
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 7 . https://qanswer-frontend.univ-st-etienne.fr/
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 8 . Writing a good review
. Using templates and paraphrases
. Graph-based answering for complex questions
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 9 . Do new research
. Write an article
. Include references to useful articles
. Communicate to publisher (conference/journal)
. Article is peer-reviewed (typically three)
. Article is accepted or rejected for publication
. If accepted, the references contribute to citation counts of respective papers
. Review process is critical to advancement of scientific community!!
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 10 . Write in a structured manner (review form) . Positives and negatives
. Concise . Summary: Unbiased . Reduce redundancy . Problem/motivation . Pinpoint to section, quote numbers . Method . Illustrate with examples . Evaluation . Attention to detail . Do not copy from abstract paraphrase! . Passive, 3rd person, polite . . Some but not too many first persons . Show understanding of article . Constructive
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 11 . Focus areas . No nitpicking
. Motivation . Do not find flaws in (or praise) future work! . Method . Do not point out too many grammar and . Evaluation spelling issues . Stay objective (I hate templates!) . Clarity, presentation, reproducibility very . Try to position paper properly important
. Look at the bigger picture: main .
advantages and disadvantages . Be diplomatic but take a stand
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 12 How can we answer questions on open knowledge graphs?
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 13 . Paraphrase-Driven Learning for Open Question Answering
. Fader et al.
. ACL 2013
. https://www.aclweb.org/anthology/P13-1158.pdf
. Answering Complex Questions by Joining Multi-Document Evidence with Quasi Knowledge Graphs
. Lu et al.
. SIGIR 2019
. https://arxiv.org/pdf/1908.00469.pdf
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 14 Paraphrase-Driven Learning for Open Question Answering
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 15 . Wikidata, YAGO, Freebase, Dbpedia
. Predicate and classes moderated
. Crowdsourced, or from Wikipedia infoboxes
. Contains selected but clean information
. Entities, predicates, classes, literals canonicalized
. Moderated schema (predicate signatures, class ontology)
. Intricate construction process
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 16 . Open vocabulary knowledge graphs
. NELL: http://rtw.ml.cmu.edu/rtw/
. Automatically extracted from text (fast)
. Reverb, Stanford OpenIE
. IE course?
. Higher coverage
. No canonicalization, noisy
. No moderated schema (bornIn city/date)
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 17 . Leverages paraphrases from WikiAnswers
. Runs on OKGs
. End-to-end system
. Factoid questions
. Open-domain QA
. Uses distant supervision
. No direct manual annotation!
. Single relation queries
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 18 . ReVerb extractions from ClueWeb09
. http://reverb.cs.washington.edu/
. https://www.lemurproject.org/clueweb09.php/
. 6B triples
. Subset used: 15M
. 600k relations, 2M entities
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 19 . Familiar terms! authored(milne, winnie-the-pooh)
. Tuple store treat(bloody-mary, hangover-symptoms)
. Patterns, templates, lexical structures
. Lexicons
. Derivations
. Lexical equivalences
. Executable queries
. Database concepts
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 20 . Problem
. Learn a question-query mapping function
. Model
. Lexicon
. Linear ranking function
. Learning
. Lexicon induction
. Parameter learning
. Evaluation
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 21 . Lexicon and derivations
. Linear ranking function
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 22 . Entity patterns
. Relation patterns
. Question patterns
. Derivation in two steps
. Match question to template
. Map surface forms to DB entries
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 23 . Multiple queries from single question
. Lots of noise
. Score queries with linear function
. Create feature vectors for question query pairs
. Learn parameter vector (weights)
. Prune by N-best if necessary
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 24 . Lexical learning
. Parameter learning
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 25 . For generalizing from initial lexicon
. Given:
. Paraphrase pair
. Derivation of one question
. Word alignment
. Produces
. New lexicon entries
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 26 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 27 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 28 Returns all triples
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 29 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 30 . Filter out noise
. Create feature vectors
. Perceptron algorithm https://en.wikipedia.org/wiki/Perceptron# Learning_algorithm
. Get top-k queries and execute!
. Leads to answer ranking!
Question Answering Systems Saarland University, Summer Semester 2020 12 May 2020 31 Answering Complex Questions by Joining Multi- Document Evidence with Quasi Knowledge Graphs
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 32 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 33 Question Answering Systems 34 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 35 Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 36 . Complexity in information needs
. Works only for simple questions
. Misses additional conditions
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 37 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Answers: Inception, Interstellar
. KGs are inherently incomplete
. Answer from text (+ KG)
. Join evidence from multiple documents on-the-fly
. Hardly any training data: develop unsupervised approach
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 38 Graph construction 1. Construct noisy quasi-KG on-the-fly
2. Augment with alignments and types
3. Spot question cornerstones
4. Compute Group Steiner Tree Graph algorithm 5. Rank answer candidates
QUEST: Question Answering with Steiner Trees
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 39 Question: Which Nolan films won an Oscar but missed a Golden Globe?
The 2011 Oscar award just announced that Inception is the winner of the Best Sound Editing award. Other winners of the day...
Nolan directed the movie Inception and other science thrillers...
Inception narrowly lost to The Social Network for Best Screenplay at the 68th Golden Globe Awards, which were declared in the afternoon..
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 40 Question: Which Nolan films won an Oscar but missed a Golden Globe?
The 2011 Oscar award just announced that Inception is the winner of the Best Sound Editing award. Other winners of the day...
Nolan directed the movie Inception and other science thrillers...
Inception narrowly lost to The Social Network for Best Screenplay at the 68th Golden Globe Awards, which were declared in the afternoon..
Question Answering Systems 19 May 2020 41 Question: Which Nolan films won an Oscar but missed a Golden Globe? Stanford AI2 MPII . Use Open Information Extraction
. Existing approaches (Stanford Open IE, ReVerb, ClausIE) not suited
. Use own noisy extractor:
.
.
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 42
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 43 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan won Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network Best Actor winner of 44 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan won Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Form noisy quasi-KG that joins Golden Globes evidence from multiple documents 83rd Academy Awards announced lost to on-the-flyThe movie Inception declared at Best Screenplay nominated The Social Network Best Actor winner of 45 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan won Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network Best Actor winner of 46 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 47 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announcedAdd types: director of ★ HearstInception patternsedited by missed out 2011 Oscars X and other
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 49 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 50 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won Insert alignment edges type Golden Globe Awards announced directed using lexiconsdirector and ofsimilarity Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 51 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 52 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 53 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Simplify by considering edges as Golden Globes homogeneous and undirected 83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 54 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 55 Question: Which Nolan films won an Oscar but missed a Golden Globe? 0.3 1.0 0.9 0.6 Best Sound Nolan Chris Nolan science thriller 1.0 0.1 won 0.4 0.3 type Golden Globe Awards directed announced director of 0.9 0.2 1.0 1.0 Inception edited by missed out 2011 Oscars 0.2 1.0 Golden Globes 0.9 83rd Academy Awards 0.4 0.9 0.2 announced 0.4 lost to The movie Inception 0.3 declared at 0.4 Best Screenplay nominated 0.1 0.1 The Social Network 0.3 0.9 1.0 type Best Actor film winner of 56 Question: Which Nolan films won an Oscar but missed a Golden Globe? 0.3 1.0 0.9 0.6 Best Sound Nolan Chris Nolan science thriller 1.0 0.1 won 0.4 0.3 type Golden Globe Awards directed announced director of 0.9 0.2 1.0 1.0 Inception edited by missed out 2011 Oscars Compute node weights0.2 using 1.0 Golden Globes 0.9 similarity with question words 83rd Academy Awards 0.4 0.9 0.2 announced 0.4 lost to The movie Inception 0.3 declared at 0.4 Best Screenplay nominated 0.1 0.1 The Social Network 0.3 0.9 1.0 type Best Actor film winner of 57 Question: Which Nolan films won an Oscar but missed a Golden Globe? 0.3 1.0 0.9 0.6 Best Sound Nolan Chris Nolan science thriller 1.0 0.1 won 0.4 0.3 type Golden Globe Awards directed announced director of 0.9 0.2 1.0 1.0 Inception edited by missed out 2011 Oscars 0.2 1.0 Golden Globes 0.9 83rd Academy Awards 0.4 0.9 0.2 announced 0.4 lost to The movie Inception 0.3 declared at 0.4 Best Screenplay nominated 0.1 0.1 The Social Network 0.3 0.9 1.0 type Best Actor film winner of 58 Question: Which Nolan films won an Oscar but missed a Golden Globe? 1.0 0.9 0.6 Best Sound 1.0 Nolan Chris Nolan science thriller won type Golden Globe Awards announced directed 0.9 director of 1.0 1.0 Inception edited by missed out 2011 Oscars 1.0 Golden Globes 0.9 83rd Academy Awards 0.9 announced lost to The movie Inception declared at Best Screenplay nominated The Social Network 0.9 1.0 type Best Actor film winner of 59 Question: Which Nolan films won an Oscar but missed a Golden Globe? 1.0 0.9 0.6 Best Sound 1.0 Nolan Chris Nolan science thriller won type Golden Globe Awards announced directed 0.9 director of 1.0 1.0 Inception edited by missed out 2011 Oscars 1.0 Identify cornerstones by Golden Globes 0.9 83rd Academy Awardsthresholding node weights 0.9 announced lost to The movie Inception declared at Best Screenplay nominated The Social Network 0.9 1.0 type Best Actor film winner of 60 Question: Which Nolan films won an Oscar but missed a Golden Globe? 1.0 0.9 0.6 Best Sound 1.0 Nolan Chris Nolan science thriller won type Golden Globe Awards announced directed 0.9 director of 1.0 1.0 Inception edited by missed out 2011 Oscars 1.0 Golden Globes 0.9 83rd Academy Awards 0.9 announced lost to The movie Inception declared at Best Screenplay nominated The Social Network 0.9 1.0 type Best Actor film winner of 61 Question: Which Nolan films won an Oscar but missed a Golden Globe? 1.0 0.9 0.6 Best Sound 1.0 Nolan Chris Nolan science thriller won type Golden Globe Awards announced directed 0.9 director of 1.0 1.0 Inception edited by missed out 2011 Oscars 1.0 Golden Globes 0.9 83rd Academy Awards 0.9 announced lost to The movie Inception declared at Best Screenplay nominated The Social Network 0.9 1.0 type Best Actor film winner of 62 Question: Which Nolan films won an Oscar but missed a Golden Globe? 1.0 0.9 0.6 Best Sound 1.0 Nolan Chris Nolan science thriller won type Golden Globe Awards announced directed 0.9 director of 1.0 1.0 Inception edited by missed out 2011 Oscars 1.0 Golden Globes 0.9 83rd Academy Awards 0.9 announced lost to The movie Inception declared at Best Screenplay nominated The Social Network Cornerstones appear in groups 0.9 1.0 type Best Actor film winner of 63 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 64 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 65 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Answers lie on paths Best Screenplay nominated The Social Network connectingtype cornerstones Best Actor film winner of 66 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 67 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 68 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes Shorter paths are better 83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 69 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 70 Question: Which Nolan films won an Oscar but missed a Golden Globe? 0.9 Best Sound 0.9 Nolan Chris Nolan 1.0 science thriller won 0.8 0.9 0.8 0.7 type Golden Globe Awards directed announced 0.8 director of 0.9 0.8 0.8 Inception edited by missed out 0.9 0.8 1.0 2011 Oscars 0.6 0.4 Golden Globes 0.9 0.4 0.7 0.9 0.5 1.0 83rd Academy Awards 0.9 announced 0.8 lost to The movie Inception 0.8 0.6 declared at Best Screenplay nominated 0.8 0.3 1.0 The Social Network 0.7 1.0 type Best Actor film 0.8 winner of 71 Question: Which Nolan films won an Oscar but missed a Golden Globe? 0.9 Best Sound 0.9 Nolan Chris Nolan 1.0 science thriller won 0.8 0.9 0.8 0.7 type Golden Globe Awards directed announced 0.8 director of 0.9 0.8 0.8 InsertInception edgeedited weights: by missed out 0.9 0.8 1.0 2011 Oscars 0.6 0.4 Golden Globes 0.9 ★ Word0.4 proximities in documents0.7 0.9 0.5 1.0 83rd Academy Awards 0.9 ★ Lexicon similarities announced 0.8 lost to The movie Inception 0.8 0.6 declared at Best Screenplay nominated 0.8 0.3 1.0 The Social Network 0.7 1.0 type Best Actor film 0.8 winner of 72 Question: Which Nolan films won an Oscar but missed a Golden Globe? 0.9 Best Sound 0.9 Nolan Chris Nolan 1.0 science thriller won 0.8 0.9 0.8 0.7 type Golden Globe Awards directed announced 0.8 director of 0.9 0.8 0.8 Inception edited by missed out 0.9 0.8 1.0 2011 Oscars 0.6 0.4 Golden Globes 0.9 0.4 0.7 0.9 0.5 1.0 83rd Academy Awards 0.9 announced 0.8 lost to The movie Inception 0.8 0.6 declared at Best Screenplay nominated 0.8 0.3 1.0 The Social Network 0.7 1.0 type Best Actor film 0.8 winner of 73 Question: Which Nolan films won an Oscar but missed a Golden Globe? 0.9 Best Sound 0.9 Nolan Chris Nolan science thriller won 0.8 0.8 0.7 type Golden Globe Awards directed announced 0.8 director of 0.9 0.8 Inception edited by missed out 2011 Oscars 0.6 0.4 Golden Globes 0.9 0.4 83rd Academy Awards announced 0.8 lost to The movie Inception declared at Best Screenplay nominated 0.8 0.3 The Social Network type Best Actor film winner of 74 Question: Which Nolan films won an Oscar but missed a Golden Globe? 0.9 Best Sound 0.9 Nolan Chris Nolan science thriller won 0.8 0.8 0.7 type Golden Globe Awards directed announced 0.8 director of 0.9 0.8 Inception edited by missed out 2011 Oscars 0.6 0.4 Golden Globes 0.9 0.4 83rd Academy Awards announced 0.8 lost to The movie Inception declared at Best Screenplay nominated 0.8 0.3 The Social Network Highertype weight paths are better Best Actor film winner of 75 . Quasi-KG:
Nodes and edges Mapping nodes and Mapping nodes and edges to weights edges to types
. Cornerstone groups: Set of sets
. At least one instance from each group is necessary in optimal subgraph
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 76 . Answers on paths connecting cornerstones
. Cost of path = Sum of edge costs (1 - edge weights)
. Higher weight paths have lower costs
. Shorter paths have lower costs
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 77 . Given:
. Undirected and weighted graph G
. Subset of vertices = Terminals T1, T2 Tn
. Find: Minimum weight tree containing all terminals
. Cornerstones = Terminals
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 78 . Terminals occur as groups Ci (cornerstones)
. Group Steiner Tree contains at least one terminal from each group
. Compute Group Steiner Trees on Quasi-KG
. We use method from Ding et al. [2007]
. Dynamic programming exponential in #terminals but O(n log n) in graph size
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 79 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan Chris Nolan science thriller won type Golden Globe Awards directed announced director of Inception edited by missed out 2011 Oscars Golden Globes
83rd Academy Awards announced lost to The movie Inception declared at Best Screenplay nominated The Social Network type Best Actor film winner of 80 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan won announced directed
Inception 2011 Oscars Golden Globes
announced lost to Best Screenplay The Social Network type film winner of 81 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound Nolan . Answers are nodes won . Cornerstones are not answers announced directed . Only entities . Must respect type constraints Inception 2011 Oscars Golden Globes
announced lost to Best Screenplay The Social Network type film winner of 82 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound . Answers are nodes won . Cornerstones are not answers announced directed . Only entities . Must respect type constraints Inception
announced lost to Best Screenplay The Social Network type winner of 83 Question: Which Nolan films won an Oscar but missed a Golden Globe?
Best Sound . Answers are nodes . Cornerstones are not answers . Only entities . Must respect type constraints Inception
Best Screenplay The Social Network Question: Which Nolan films won an Oscar but missed a Golden Globe?
. Answers are nodes . Cornerstones are not answers . Only entities . Must respect type constraints Inception
The Social Network Question: Which Nolan films won an Oscar but missed a Golden Globe?
. Answers ranked by multiple criteria . Best answer chosen
Inception Sum of node weights 8.56 Sum of node weights 9.75 Number of GSTs 2 Number of GSTs 5 Sum of tree costs 7.98 Sum of tree costs 6.78 Distance to cornerstones 27 Distance to cornerstones 19 The Social Network Question: Which Nolan films won an Oscar but missed a Golden Globe?
. Answers ranked by multiple criteria . Best answer chosen
Inception
Sum of node weights 9.75 Number of GSTs 5 Sum of tree costs 6.78
Distance to cornerstones 19 . Benchmark: 300 complex questions
. Metric: Hit scores (correct answer in top-5)
. Baselines: QA algorithms and graph methods
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 88 QUEST outperforms others
Question Answering Systems 19 May 2020 89 Cannot join multi- doc evidence
Question Answering Systems 19 May 2020 90 Cannot handle complexity
Question Answering Systems 19 May 2020 91 Steiner trees help
Question Answering Systems 19 May 2020 92 . Which Japanese baseball player was contracted for Los Angeles Angels who also
played for Hokkaido Nippon-ham fighters?
. Which aspiring model split with Chloe Moretz and is dating Lexi Wood?
. Where did Sylvie Vartan meet her future husband Johnny Hallyday?
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 93 . Answers complex questions directly over text (+KG)
. On-the-fly joining of multi-document evidence
. Robust to high degrees of noise
. Robust to ungrammatical constructs
. No need for question decomposition
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 94 1. Diversity in question formulation
2. Complexity in information needs
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 95 . State-of-the-art techniques for QA
. Open KGs are noisy but useful
. Paraphrase-driven learning is vital
. Utility of templates goes beyond curated KGs
. Graph-based methods are powerful for complex questions
. Group Steiner trees on noisy quasi-KGs enables answering complex questions
Find out more at qa.mpi-inf.mpg.de
Question Answering Systems Saarland University, Summer Semester 2020 Rishiraj Saha Roy 19 May 2020 96