Retrieve, Read, Rerank, then Iterate: Answering Open-Domain Questions of Varying Reasoning Steps from Text Peng Qi*♠♥ Haejun Lee*| Oghenetegiri “TG” Sido*♠ Christopher D. Manning♠ ♠ Computer Science Department, Stanford University ~ JD AI Research | Samsung Research {pengqi, osido, manning}@cs.stanford.edu,
[email protected] Abstract Despite this success, most previous systems are developed with, and evaluated on, datasets that We develop a unified system to answer di- contain exclusively single-hop questions (ones that rectly from text open-domain questions that require a single document or paragraph to answer) may require a varying number of retrieval steps. We employ a single multi-task trans- or multi-hop ones. As a result, their design is often former model to perform all the necessary tailored exclusively to single-hop (e.g., Chen et al., subtasks—retrieving supporting facts, rerank- 2017; Wang et al., 2018b) or multi-hop questions ing them, and predicting the answer from all (e.g., Nie et al., 2019; Min et al., 2019; Feldman retrieved documents—in an iterative fashion. and El-Yaniv, 2019; Zhao et al., 2020a; Xiong We avoid making crucial assumptions as previ- et al., 2021); even when the model is designed to ous work that do not transfer well to real-world work with both, it is often trained and evaluated on settings, including exploiting knowledge of the e.g. fixed number of retrieval steps required to an- exclusively single-hop or multi-hop settings ( , swer each question or using structured meta- Asai et al., 2020). In practice, not only can we data like knowledge bases or web links that not expect open-domain QA systems to receive have limited availability.