Answering Complex Open-domain Questions Through Iterative Query Generation Peng Qiy Xiaowen Lin∗† Leo Mehr∗† Zijian Wang∗‡ Christopher D. Manningy y Computer Science Department z Symbolic Systems Program Stanford University fpengqi, veralin, leomehr, zijwang,
[email protected] Q: Which novel by the author of Search Results with queries derived from the Abstract original question “Armada” will be adapted as a feature film by Steven Spielberg? Which novel by the author of “Armada” will be It is challenging for current one-step retrieve- adapted as a feature film by Steven Spielberg? and-read question answering (QA) systems to A: Ready Player One The collector The Color Purple (film) answer questions like “Which novel by the au- Armada (novel) Kim Wozencraft thor of ‘Armada’ will be adapted as a feature Armada is a science fiction novel by novel by the author of “Armada” Ernest Cline, … Armada (novel) film by Steven Spielberg?” because the ques- Author, Author (novel) tion seldom contains retrievable clues about Ernest Cline Armada the missing entity (here, the author). Answer- Ernest Christy Cline … co-wrote the Armada author screenplay for the film adaptation of Armada ing such a question requires multi-hop reason- Ready Player One, directed by Armada Centre ing where one must gather information about Stephen Spielberg. Halley Armada the missing entity (or facts) to proceed with Figure 1: An example of an open-domain multi-hop further reasoning. We present GOLDEN (Gold Entity) Retriever, which iterates between read- question from the HOTPOTQA dev set, where “Ernest ing context and retrieving more supporting Cline” is the missing entity.