Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents Shunyu Yaoy∗ Karthik Narasimhany Matthew Hausknechtz yPrinceton University zMicrosoft Research {shunyuy, karthikn}@princeton.edu
[email protected] Abstract et al., 2020), open-domain question answering systems (Ammanabrolu et al., 2020), knowledge Text-based games simulate worlds and inter- act with players using natural language. Re- graphs (Ammanabrolu and Hausknecht, 2020; Am- cent work has used them as a testbed for manabrolu et al., 2020; Adhikari et al., 2020), and autonomous language-understanding agents, reading comprehension systems (Guo et al., 2020). with the motivation being that understanding Meanwhile, most of these models operate un- the meanings of words or semantics is a key der the reinforcement learning (RL) framework, component of how humans understand, reason, where the agent explores the same environment and act in these worlds. However, it remains in repeated episodes, learning a value function or unclear to what extent artificial agents utilize semantic understanding of the text. To this policy to maximize game score. From this per- end, we perform experiments to systematically spective, text games are just special instances of reduce the amount of semantic information a partially observable Markov decision process available to a learning agent. Surprisingly, we (POMDP) (S; T; A; O; R; γ), where players issue find that an agent is capable of achieving high text actions a 2 A, receive text observations o 2 O scores even in the complete absence of lan- and scalar rewards r = R(s; a), and the under- guage semantics, indicating that the currently lying game state s 2 S is updated by transition popular experimental setup and models may 0 be poorly designed to understand and leverage s = T (s; a).